Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker version manual? #13

Open
devcomputer opened this issue Apr 13, 2024 · 7 comments
Open

docker version manual? #13

devcomputer opened this issue Apr 13, 2024 · 7 comments

Comments

@devcomputer
Copy link

devcomputer commented Apr 13, 2024

i run your docker inside an lxc and when I run docker-compose up i see a never ending list of crawls to iana and rfc-editor and so on. no idea how long this is going to take but its quite a while now and I have no idea if this is part of the first start or if need to pass any arguments on start.
sth. like "go edit pryingdeep.yaml first"?

@iudicium
Copy link
Owner

iudicium commented Apr 17, 2024

It's not part of the first start. The crawler finds them as it goes.
Could you share how some of the infinite links look like for e.g iana or rfc-editor?
A quick fix right now would be to add these infinite websites to dissalowed-domains in pryingdeep.yaml

@fabstest
Copy link

fabstest commented May 8, 2024

I think he is referring to this one. I encounter the same issue

image

@iudicium
Copy link
Owner

iudicium commented May 9, 2024

I think he is referring to this one. I encounter the same issue

image

Oh that happens because the default url in the config
Is https://example.com
Remove it and it won't go to iana anymore or pass new urls with this arg

@fabstest
Copy link

fabstest commented May 9, 2024

After removing it I'll just run docker compose up then it'll be fine? Or I still need to rebuild?

@devcomputer
Copy link
Author

devcomputer commented Jun 18, 2024

i tried some debian based machines but they all also fail using the docker-compose.yml

image

when I uncomment the healthcheck the containers start and when editing the config to another URL it all just crashed again...but with new errors.
¯_(ツ)_/¯

@iudicium
Copy link
Owner

After removing it I'll just run docker compose up then it'll be fine? Or I still need to rebuild?

Yeah, you need to rebuild. Make sure you give a starting link if you're using the crawl command

@iudicium
Copy link
Owner

iudicium commented Jun 20, 2024

i tried some debian based machines but they all also fail using the docker-compose.yml

image

when I uncomment the healthcheck the containers start and when editing the config to another URL it all just crashed again...but with new errors. ¯_(ツ)_/¯

Maybe there's some issue with the healthcheck that I'm doing here.
Make sure you re-build with the new urls though. I'll try to add some explanation to Docker section within a few days and check the healthcheck.
For now, can you paste the traceback of the new errors here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants