Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

训练mag240M数据的metapath2vec #461

Open
d049 opened this issue Sep 20, 2022 · 4 comments
Open

训练mag240M数据的metapath2vec #461

d049 opened this issue Sep 20, 2022 · 4 comments

Comments

@d049
Copy link

d049 commented Sep 20, 2022

需要什么样的机器配置，才能训练mag240M数据的metapath2vec，有接近3���个节点，embeding非常大，训练速度很慢
@d049
Copy link
Author

d049 commented Sep 20, 2022

而且显存占用也很大，batch=1都难以训练
@Yelrose
Copy link
Collaborator

Yelrose commented Sep 22, 2022 •
edited
Loading

metapath2vec的训练建议使用分布式CPU模式。

我们已经将MAG240M 训练好的 m2v_embedding 提供了云盘下载的地址。具体看下面链接里面，m2v embedding部分
https://github.com/PaddlePaddle/PGL/tree/main/examples/kddcup2021/MAG240M/r_unimp
@d049
Copy link
Author

d049 commented Sep 26, 2022

谢谢给的预训练特征，问一下分布式cpu模式，用的什么机器配置，训练了多久
@Yelrose
Copy link
Collaborator

Yelrose commented Sep 26, 2022

我们用MPI集群训练的，为了追求速度20台机一起训，因为训这套向量的时间已经比较久了，花的时间记录没了。你也可以按照 Graph4Rec里面提供的配置去训，只要机器内存加起来能超过向量所需的内存，应该能跑起来了。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@Yelrose @d049