A Multimodal Deep Learning Framework for Personalized Event Recommendation with Hyperparameter Optimization via Genetic Algorithm
DOI:
https://doi.org/10.56158/jcosml.2026.1.1.4Keywords:
event recommendation; multimodal fusion; gated multimodal unit; bayesian personalized ranking; genetic algorithm; heterogeneous graph transformer; hyperparameter optimizationAbstract
Personalized event recommendation poses a fusion challenge: textual descriptions, social graph connectivity, and user interaction history each carry information that no single-modality model can fully exploit. This paper presents a multimodal deep learning framework that combines DistilBERT text embeddings, Heterogeneous Graph Transformer (HGT) graph embeddings, and interaction features through a Gated Multimodal Unit (GMU) trained with Bayesian Personalized Ranking (BPR) loss. A Genetic Algorithm optimizes six hyperparameters including learning rate, batch size, embedding dimensions, and graph encoder depth. An order-preserving, leakage-free strategy is proposed for handling out-of-vocabulary tokens: rather than mapping unknown tokens to a shared index, they are removed from the input sequence while surviving tokens retain their original positional order, preventing representational collapse at small vocabulary sizes. Experiments are conducted on a controlled synthetic dataset comprising 10,000 users, 3,000 events, and 80,000 interactions, with random and popularity baselines included as reference points. Results confirm correct gradient flow and optimizer behavior; the framework is positioned as an architectural proof of concept intended for subsequent validation on datasets with genuine preference structure.
References
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua, "Neural collaborative filtering," in Proc. 26th International World Wide Web Conference, pp. 173-182, 2017, https://doi.org/10.1145/3038912.3052569.
Y. Hou, S. Mu, W. X. Zhao, Y. Li, B. Ding, and J.-R. Wen, "Towards universal sequence representation learning for recommender systems," in Proc. 28th ACM SIGKDD Conference, pp. 585-593, 2022, https://doi.org/10.1145/3534678.3539381.
J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, "Neural attentive session-based recommendation," in Proc. ACM CIKM, pp. 1419-1428, 2017, https://doi.org/10.1145/3132847.3132926.
X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua, "Neural graph collaborative filtering," in Proc. 42nd ACM SIGIR, pp. 165-174, 2019, https://doi.org/10.1145/3331184.3331267.
R. van den Berg, T. N. Kipf, and M. Welling, "Graph convolutional matrix completion," arXiv preprint arXiv:1706.02263, 2017, https://arxiv.org/abs/1706.02263.
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, "Graph convolutional neural networks for web-scale recommender systems," in Proc. 24th ACM SIGKDD, pp. 974-983, 2018, https://doi.org/10.1145/3219819.3219890.
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, "Modeling relational data with graph convolutional networks," in Proc. 15th ESWC, pp. 593-607, 2018, https://doi.org/10.1007/978-3-319-93417-4_38.
F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, "BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer," in Proc. 28th ACM CIKM, pp. 1441-1450, 2019, https://doi.org/10.1145/3357384.3357895.
W.-C. Kang and J. McAuley, "Self-attentive sequential recommendation," in Proc. IEEE ICDM, pp. 197-206, 2018, https://doi.org/10.1109/ICDM.2018.00035.
Y. Wei, X. Wang, L. Nie, X. He, R. Hong, and T.-S. Chua, "MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video," in Proc. 27th ACM Multimedia, pp. 1437-1445, 2019, https://doi.org/10.1145/3343031.3351034.
Q. Liu, J. Hu, Y. Xiao, X. Zhao, J. Gao, W. Wang, Q. Li, and J. Tang, "Multimodal recommender systems: A survey," ACM Computing Surveys, 2024, https://arxiv.org/abs/2302.03883.
J. Arevalo, T. Solorio, M. Montes-y-Gomez, and F. A. Gonzalez, "Gated multimodal units for information fusion," in Proc. ICLR Workshop on Representation Learning, 2017, https://arxiv.org/abs/1702.01992.
Z. Hu, Y. Dong, K. Wang, and Y. Sun, "Heterogeneous graph transformer," in Proc. The Web Conference 2020, pp. 2704-2710, 2020, https://doi.org/10.1145/3366423.3380027.
K. O. Stanley, J. Clune, J. Lehman, and R. Miikkulainen, "Designing neural networks through neuroevolution," Nature Machine Intelligence, vol. 1, no. 1, pp. 24-35, 2019, https://doi.org/10.1038/s42256-018-0006-z.
E. Real, A. Aggarwal, Y. Huang, and Q. V. Le, "Regularized evolution for image classifier architecture search," in Proc. 33rd AAAI, pp. 4780-4789, 2019, https://doi.org/10.1609/aaai.v33i01.33014780.
F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, and C. Gagne, "DEAP: Evolutionary algorithms made easy," Journal of Machine Learning Research, vol. 13, pp. 2171-2175, 2012, http://jmlr.org/papers/v13/fortin12a.html.
K. Zhou, X. Huang, Q. Song, R. Chen, and X. Hu, "Auto-GNN: Neural architecture search of graph neural networks," Frontiers in Big Data, vol. 5, 2022, https://doi.org/10.3389/fdata.2022.1029307.
J. Zhang, X. Shi, S. Zhao, and I. King, "STAR-GCN: Stacked and reconstructed graph convolutional networks for recommender systems," in Proc. 28th IJCAI, pp. 4264-4270, 2019, https://doi.org/10.24963/ijcai.2019/592.
Y. Liu, L. Chen, X. He, J. Peng, Z. Zheng, and J. Tang, "Modelling high-order social relations for item recommendation," IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 9, pp. 4385-4397, 2022, https://doi.org/10.1109/TKDE.2020.3039463.
Y. Hou, B. Hu, Z. Zhang, and W. X. Zhao, "CORE: Simple and effective session-based recommendation within consistent representation space," in Proc. 45th ACM SIGIR, pp. 1796-1801, 2022, https://doi.org/10.1145/3477495.3531955.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter," arXiv preprint arXiv:1910.01108, 2019, https://arxiv.org/abs/1910.01108.
W. Hua, Y. Ge, S. Xu, J. Ji, Z. Li, and Y. Zhang, "UP5: Unbiased foundation model for fairness-aware recommendation," in Proc. 18th EACL, pp. 1899-1912, 2024, https://doi.org/10.18653/v1/2024.eacl-long.114.
J. Xu et al., "AutoHEnsGNN: Winning solution to AutoGraph challenge for KDD Cup 2020," in Proc. 38th IEEE ICDE, pp. 352-366, 2022, https://doi.org/10.1109/ICDE53745.2022.00031.
B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, "Session-based recommendations with recurrent neural networks," in Proc. ICLR, 2016, https://arxiv.org/abs/1511.06939.
S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, "BPR: Bayesian personalized ranking from implicit feedback," in Proc. 25th UAI, pp. 452-461, 2009, https://arxiv.org/abs/1205.2618.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Serap Gülgen (Author)

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.