BERT4Rec

BERT4Rec took the BERT playbook and pointed it at shopping histories. Published at CIKM 2019 by an Alibaba team (Fei Sun et al.), it models a user's interaction sequence with deep bidirectional self-attention: every item fuses information from both its past and its future in the sequence. To keep the bidirectional model from trivially "seeing" its target, it trains with a Cloze objective — randomly masking items and predicting them from both sides — which also generates far more training samples than next-item prediction alone.

The paper argued, with experiments on four benchmark datasets, that the old left-to-right assumption in sequential models was needlessly restrictive: user behavior is not always rigidly ordered, and bidirectional context learns stronger representations. The authors released both TensorFlow and PyTorch code, and it became the canonical bidirectional baseline the way SASRec became the unidirectional one.

It is heavier than SASRec and, like all ID-based models, struggles with cold-start items — a limitation the later text-based models (UniSRec, RecFormer) were designed to fix.

• Paper-reported (CIKM 2019, arXiv 1904.06690): consistently outperformed state-of-the-art sequential models across four benchmark datasets on the standard next-item metrics
• No independent benchmark body re-scores it; its standing rests on wide reproduction as a baseline in later sequential-recommendation work