LIU Yue’s blogs
欢迎来到我的个人博客！欢迎讨论！
欢迎来到我的个人博客！欢迎讨论！
Interpretable GNNs [Draft]
Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism ( Citation: Miao, Liu & al., 2022 Miao, S., Liu, M. & Li, P. (2022). Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism. PMLR. Retrieved from https://proceedings.mlr.press/v162/miao22a.html )
Text to Video Generative Models [Draft]
本文记录文生图、文生视频的相关论文。 Breathing Life Into Sketches Using Text-to-Video Priors ( Citation: Gal, Vinker & al., 2023 Gal, R., Vinker, Y., Alaluf, Y., Bermano, A., Cohen-Or, D., Shamir, A. & Chechik, G. (2023). Breathing Life Into Sketches Using Text-to-Video Priors. https://doi.org/10.48550/arXiv.2311.13608 ) 是英伟达在CVPR2024的一篇...
Multi-modal Learning [Draft]
本文记录一些读过的多模态论文。 Learning to Prompt for Vision-Language Models ( Citation: Zhou, Yang & al., 2022 Zhou, K., Yang, J., Loy, C. & Liu, Z. (2022). Learning to Prompt for Vision-Language Models. International Journal of Computer Vision, 130(9). 2...
Graph for LLMs [Draft]
图数据（包括知识图谱）对大语言模型的优化也有帮助。 RHO ($\rho$): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding ( Citation: Ji, Liu & al., 2023 Ji, Z., Liu, Z., Lee, N., Yu, T., Wilie, B., Zeng, M. & Fung, P. (2023). RHO ($\rho$): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding. https://doi.org/10.48550/arXiv.2212.01588 ) 研究了...
LLM for Graphs [Draft]
大语言模型火起来后，图机器学习社区的研究者也开始探究大语言模型在图数据上的可能应用。 Talk like a Graph: Encoding Graphs for Large Language Models ( Citation: Fatemi, Halcrow & al., 2023 Fatemi, B., Halcrow, J. & Perozzi, B. (2023). Talk like a Graph: Encoding Graphs...
A Review of Graph Neural Networks [Draft]
本文致力于梳理常见的图节点编码器，包括较早的DeepWalk, Node2vec以及各种经典的图神经网络。 DeepWalk: Online Learning of Social Representations ( Citation: Perozzi, Al-Rfou & al., 2014 Perozzi, B., Al-Rfou, R. & Skiena, S. (2014)....
Trajactory-User Linking Problem [Draft]
S2TUL: A Semi-Supervised Framework for Trajectory-User Linking ( Citation: Deng, Sun & al., 2023 Deng, L., Sun, H., Zhao, Y., Liu, S. & Zheng, K. (2023). S2TUL: A Semi-Supervised Framework for Trajectory-User Linking. ACM. https://doi.org/10.1145/3539597.3570410 ) 提出了轨迹-用户匹配问题的半监督方法。 数据集：Foursquare 作者...
Optimizers in Machine Learning [Draft]
本文梳理常用的优化器算法。 Adam: A Method for Stochastic Optimization ( Citation: Kingma & Ba, 2017 Kingma, D. & Ba, J. (2017). Adam: A Method for Stochastic Optimization. https://doi.org/10.48550/arXiv.1412.6980 ) 提出了Adam优化器算法。
LLM [Draft]
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining 研究动机 – 数据混合比例 大模型的训练数据是多种来源混合的。例如，著名的The-pile数据集，包含了24%的网页数据、9%的...
Knowledge Graph Reasoning [Draft]
论文发现于 Awesome-Knowledge-Graph-Reasoning Github 仓库地址。 InGram: Inductive Knowledge Graph Embedding via Relation Graphs ( Citation: Lee, Chung & al., 2023 Lee, J., Chung, C. & Whang, J. (2023). InGram: Inductive Knowledge Graph Embedding via Relation Graphs. https://doi.org/10.48550/arXiv.2305.19987 ) 研究了知识图谱的归纳式推理问题。 知识图谱 知识图谱是一种有...
Recurrent Neural Networks
根据其他博客1和博客2梳理循环神经网络。 Recurrent Neural Network (RNN) RNN介绍 回顾多层感知机网络，每个输入只对应一个输出。当输入的数据是一个序列，且序列中的元素...
Anti-Money Laundering [Draft]
反洗钱是异常检测的一个子问题。 Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics ( Citation: Weber, Domeniconi & al., 2019 Weber, M., Domeniconi, G., Chen, J., Weidele, D., Bellei, C., Robinson, T. & Leiserson, C. (2019). Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics. https://doi.org/10.48550/arXiv.1908.02591 ) 贡献了当时最大的反...
Beyond Message-Passing GNNs [Draft]
Representing Long-Range Context for Graph Neural Networks with Global Attention ( Citation: Wu, Jain & al., 2022 Wu, Z., Jain, P., Wright, M., Mirhoseini, A., Gonzalez, J. & Stoica, I. (2022). Representing Long-Range Context for Graph Neural Networks with Global Attention. https://doi.org/10.48550/arXiv.2201.08821 ) 提出了GraphTrans，一种使用Transformer拓...
Attention Mechanism
Attention机制 根据OpenAI工程师Andrej Karpathy的讲解视频梳理Attention机制及其在GPT（Generativ...
Unsupervised Deep Graph Structure Learning
现实中图的结构可能是不完整或有噪声的。为了在图结构不可靠的情况下较好地完成下游任务，研究者提出了如下的图结构学习算法。 Towards Unsupervised Deep Graph Structure Learning 论文链接：...
Generalization of GNNs and MLPs
本文是 ( Citation: Xu, Zhang & al., 2021 Xu, K., Zhang, M., Li, J., Du, S., Kawarabayashi, K. & Jegelka, S. (2021). How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks. https://doi.org/10.48550/arXiv.2009.11848 ) 的论文解读。OpenReview显示这篇论文是ICLR2021的Oral论...
Stochastic Blockmodels [Draft]
本文基于 ( Citation: Karrer & Newman, 2011 Karrer, B. & Newman, M. (2011). Stochastic blockmodels and community structure in networks. Physical Review E, 83(1). 016107. https://doi.org/10.1103/PhysRevE.83.016107 ) 介绍社区发现中经典的随机块模型。 随机块模型是一种生成模型，它建模了社区与图生成之间的联系。尽...
Structural Community Detection
本文专注于整理社区发现的方法。 问题定义 给定一个图$G=(V,E)$，找到一个映射$g:V\to {1,2,\cdots,K}$，$g$将图中的...
GNN and MLP
最近发现一篇ICLR2023 spotlight的蒸馏GNN到MLP的论文 ( Citation: Tian, Zhang & al., 2023 Tian, Y., Zhang, C., Guo, Z., Zhang, X. & Chawla, N. (2023). NOSMOG: Learning Noise-robust and Structure-aware MLPs on Graphs. https://doi.org/10.48550/arXiv.2208.10010 ) ，觉得很新鲜。向...
VAE
Variational Autoencoders 原博主为Lilian Weng 与简单的自编码器不同，变分自编码器的表征$\mathbf{z}$是一个分布。 给定一个数据集$\mathbf{X}=...
FlashAttention
FlashAttention论文发表于Neurips2022，第一单位是斯坦福大学。 作者提出了一种使用更小代价计算self-attentio...