Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make an action for checking proxies #8

Open
t0int1337 opened this issue Sep 17, 2023 · 9 comments
Open

make an action for checking proxies #8

t0int1337 opened this issue Sep 17, 2023 · 9 comments

Comments

@t0int1337
Copy link

would be nice

@t0int1337 t0int1337 changed the title make a action for checking proxies Sep 17, 2023
@MuRongPIG
Copy link
Owner

I have to admit that it is absolutely a good idea,but im worrying that github might regard the action as abuse.:(

@leoasah2047
Copy link

I think the best way to check the proxies is to create github actions which take into account the limit set by them, which is 6hrs for the Job execution time.

I tried testing it on pythonanywhere and it takes roughly 22hrs to run the checkproxy.py file.

Taking that into account, this may need that you create three (or maybe more) different checkproxy.py files for each to work on socks4 , socks5 and http, and creating actions to run each file

Note that sock5 and socks4 tend to have 10k proxies each on every run by getproxy.py, whilst http has about 20k or so, thus in order to create the needed checkproxy.py files, you may need to create a function in each newly created checkproxy.py to extract just the needed amount of proxies (about 5K per .txt), so as to meetup with the 6hrs limits set by github actions.

@huolongguo1O
Copy link

Well there are some bugs in checkproxy.py I think

@MuRongPIG
Copy link
Owner

I think the best way to check the proxies is to create github actions which take into account the limit set by them, which is 6hrs for the Job execution time.

I tried testing it on pythonanywhere and it takes roughly 22hrs to run the checkproxy.py file.

Taking that into account, this may need that you create three (or maybe more) different checkproxy.py files for each to work on socks4 , socks5 and http, and creating actions to run each file

Note that sock5 and socks4 tend to have 10k proxies each on every run by getproxy.py, whilst http has about 20k or so, thus in order to create the needed checkproxy.py files, you may need to create a function in each newly created checkproxy.py to extract just the needed amount of proxies (about 5K per .txt), so as to meetup with the 6hrs limits set by github actions.

I have run checkproxy.py on Github Action before,but it didn't return any respond before I run out of my patience. However, the program works on my server while it just takes about 10 minutes (i dont know why)

@MuRongPIG
Copy link
Owner

MuRongPIG commented Dec 2, 2023

maybe u can try other checker programs like https://github.com/monosans/proxy-scraper-checker .
If i have enough time i'll try to improve it and fix bugs.thanks for your ideas!

@huolongguo1O
Copy link

I think the best way to check the proxies is to create github actions which take into account the limit set by them, which is 6hrs for the Job execution time.
I tried testing it on pythonanywhere and it takes roughly 22hrs to run the checkproxy.py file.
Taking that into account, this may need that you create three (or maybe more) different checkproxy.py files for each to work on socks4 , socks5 and http, and creating actions to run each file
Note that sock5 and socks4 tend to have 10k proxies each on every run by getproxy.py, whilst http has about 20k or so, thus in order to create the needed checkproxy.py files, you may need to create a function in each newly created checkproxy.py to extract just the needed amount of proxies (about 5K per .txt), so as to meetup with the 6hrs limits set by github actions.

I have run checkproxy.py on Github Action before,but it didn't return any respond before I run out of my patience. However, the program works on my server while it just takes about 10 minutes (i dont know why)

In my server, it runs at least 30 minutes(I ctrl-c-ed), but it used to be about 10 minutes too...
I'm sure that it checked http/https two times.
I wrote a checker in c and it's very fast ;-) May I create a pull request this weekend?

@MuRongPIG
Copy link
Owner

我认为检查代理的最佳方法是创建 github 操作,其中考虑了它们设置的限制,即作业执行时间的 6 小时。我尝试在 pythonanywhere 上测试它,运行 checkproxy.py 文件大约需要 22 小时。考虑到这一点,这可能需要您为每个文件创建三个(或更多)不同的 checkproxy.py 文件才能在 socks4socks5http 上运行,并创建操作来运行每个文件 请注意,sock5 和 socks4getproxy.py 每次运行时往往各有 10k 个代理,而 http 大约有 20k 个左右, 因此,为了创建所需的 checkproxy.py 文件,您可能需要在每个新创建的 checkproxy.py 中创建一个函数,以提取所需的代理数量(每.txt约 5K),以满足 GitHub Actions 设置的 6 小时限制。

我以前在 Github Action 上运行过 checkproxy.py,但在我耗尽耐心之前它没有返回任何响应。但是,该程序在我的服务器上运行,只需大约 10 分钟(我不知道为什么)

在我的服务器中,它至少运行了 30 分钟(我 ctrl-c-ed),但以前也大约是 10 分钟......我敢肯定它检查了两次 http/https。我在 c 中写了一个检查器,它非常快;-)我可以在本周末创建拉取请求吗?

of cource u can. thanks!<3

@0x24a
Copy link
Contributor

0x24a commented Jun 9, 2024

the checker in my pr only takes 5min&~100MB RAM Usage on a 1c1g machine to check 63975 proxies

@t0int1337
Copy link
Author

@MuRongPIG

@0x24a 's checker looks fine but can it work with GitHub actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
5 participants