Paper Notes: Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommenda

Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation

• LINK: https://doi.org/10.24963/ijcai.2019/585

• CLASSIFICATION: RECOMMENDER-SYSTEM, SEQUENTIAL RECOMMENDER, LONG SHORT INTEREST

• YEAR: Submitted on 24 May 2019

• FROM: IJCAI 2019

• WHAT PROBLEM TO SOLVE: Previous approaches neglect the importance of dynamically integrating long-term and short-term user modeling paradigms. Moreover, users’ behaviors are much more complex than sentences in language modeling or images in visual computing, thus the classical structures of RNN such as Long Short-Term Memory (LSTM) need to be upgraded for better user modeling.

• SOLUTION: In this paper, we improve the traditional RNN structure by proposing a time-aware controller and a content-aware controller, so that contextual information can be well considered to control the state transition. We further propose an attention-based framework to combine users’ long-term and short-term preferences, thus users’ representation can be generated adaptively according to the specific context. We conduct extensive experiments on both public and industrial datasets. The results demonstrate that our proposed method outperforms several state-of-art methods consistently.

• CORE POINT:

  • Two Key Problems

    1. Dynamic Time Intervals: Intuitively, two actions within a short time interval tend to share a closer relationship than two actions that within a long time interval. Thus this kind of temporal distance deserve special handling.
    2. Dynamic Latent Intent: Customer intent, also known as the user’s main purpose behind his/her behavior, is often changing from session to session. Irrelevant actions are useless for predicting a user’s certain future action.

  • Short-Term Modeling

    • Original LSTM

      

    • Time irregularity

      To tackle the problem of time irregularity, we modify the gating logic in LSTM to make it sensitive to time changes.
      We introduce two time-aware features, i.e., time interval feature δ t k δ_{t_k} δtk​​and time span feature s t k s_{t_k} stk​​, then compute time gates ( T δ T_δ Tδ​ and T s T_s Ts​) accordingly, as follows:

      

    • Semantic irregularity

      To tackle the problem of semantic irregularity, we adopt attentive mechanisms to suppress the information that devi- ates from the target direction.

      An item’s attention score is computed by:

      

      To adjust the cell and hidden states:

      

      Formulate user’s short term representa- tion as the weighted average of all the hidden states:

      

  • Long-Term Modeling

    The weighting score is computed as:

    

    Users are represented through the items that they interacted with:

    

  • Adaptive Fusion Approach

    The motivation is which component should play a more important role is determined by the specific context, such as when (if next action is taken shortly after the previous behaviors, then short-term preference may be more informative) and what (some categories of items such as mobile phone are better inferred from long-term preference, while some categories such as mobile accessories are better inferred from short-term information).

    

    Note that for contextual information, we can include various kinds of important features according to what we have at hand, such as time interval, time stamp, location, and target item category. In our experiments we only include the item interval and the prediction embedding of the target item.

  • Optimization

    

    where Θ denotes the set of trainable parameters.

  • Experiments

    • Datasets

      Amazon dataset: Electronics, Movies and TV, CDs and Vinyl, as well as a subset of the entire merged Amazon dataset.

      Industrial dataset: Mircosoft display personalized advertisements on MSN homepage in a fashion that advertisements look like part of the editorial flow of the page.

      

    • Results

      • AUC

        

      • F1-Score

        

• EXISTING PROBLEMS: The architecture of long-term modeling is simple, propably can add a slightly more complex architecture.

• IMPROVEMENT IDEAS: Fuse the following papaers:

  1. Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems
  2. Hi-Fi Ark: Deep User Representation via High-Fidelity Archive Network