Reasoning-Augmented Occupation Prediction Framework

논문 컨텍스트

Paper title: On Reasoning Behind Next Occupation Recommendation Abstract: In this work, we develop a novel reasoning approach to enhance the performance of large language models (LLMs) in future occupation prediction. In this approach, a reason generator first derives a ``reason'' for a user using his/her past education and career history. The reason summarizes the user's preference and is used as the input of an occupation predictor to recommend the user's next occupation. This two-step occupation prediction approach is, however, non-trivial as LLMs are not aligned with career paths or the unobserved reasons behind each occupation decision. We therefore propose to fine-tune LLMs improving their reasoning and occupation prediction performance. We first derive high-quality oracle reasons, as measured by factuality, coherence and utility criteria, using a LLM-as-a-Judge. These oracle reasons are then used to fine-tune small LLMs to perform reason generation and next occupation prediction. Our extensive experiments show that: (a) our approach effectively enhances LLM's accuracy in next occupation prediction making them comparable to fully supervised methods and outperforming unsupervised methods; (b) a single LLM fine-tuned to perform reason generation and occupation prediction outperforms two LLMs fine-tuned to perform the tasks separately; and (c) the next occupation prediction accuracy depends on the quality of generated reasons. Our code is available at https://github.com/Sarasarahhhhh/job_prediction. Passages referencing this figure: ecord is enriched with standardized occupational titles and 8-digit codes aligned with the O*NET-SOC 2019 taxonomy 2 2 2 https://www.onetcenter.org/taxonomy/2019/list.html . We first hold out 1,000 users as the test set. For the remaining data, we perform our data filtering procedure and obtain 3,646 high-quality users for training, resulting in a final dataset of 4,646 users. Proposed Framework. Figure 1 illustrates our proposed framework of reasoning-augmented next occupation prediction. It consists of three major phases: oracle reason generation , model training , and model inference . In the oracle reason generation phase, we build a dataset that connects each user’s education and career history with the target next occupation using a high-quality explanation text, called the oracle re erence Optimization (DPO) [ 12 , 8 ] that refines reason generation quality by aligning the model output with preferred reasons. In the model inference phase, the fine-tuned model is used to predict the next occupation for unseen users. The model takes an unseen user’s education and job history as input, generates a reasoning explanation, and subsequently predicts the most likely next occupation. Figure 1 : Reasoning-Augmented Occupation Prediction Framework. 4 Oracle Reason Oracle Reason Generation. A user may have many possible motivations when seeking their next occupation. The most likely reason is one that reflects the user’s preference and how this preference guides the choice of next occupation. We thus construct an oracle reason by leveraging both the user’s career history and the