Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Touble using Docker monitor Connect ENOENT /var/run/docker.sock #2061

Closed
2 tasks done
Meulator opened this issue Sep 9, 2022 · 37 comments
Closed
2 tasks done

Touble using Docker monitor Connect ENOENT /var/run/docker.sock #2061

Meulator opened this issue Sep 9, 2022 · 37 comments
Labels

Comments

@Meulator
Copy link

Meulator commented Sep 9, 2022

⚠️ Please verify that this bug has NOT been raised before.

  • I checked and didn't find similar issue

🛡️ Security Policy

📝 Describe your problem

Hello,

fairly simple issue, i installed uptime kuma using docker / docker compose, it works well, updated many times and now i'd like to try the docker monitoring to monitor the machine hosting uptime kuma and other docker containers but when i try to set it up i got this error Connect ENOENT /var/run/docker.sock

how to allow uptime kuma to connect it?

🐻 Uptime-Kuma Version

1.18.0

💻 Operating System and Arch

Ubuntu 20.04

🌐 Browser

Brave

🐋 Docker Version

docker 20.10.17

🟩 NodeJS Version

No response

@Meulator Meulator added the help label Sep 9, 2022
@louislam
Copy link
Owner

louislam commented Sep 9, 2022

By default, a docker container is self-contained and cannot access your host. I think you need to bind the /var/run/docker.sock to your container.

docker-compose:

volumes:
   - /var/run/docker.sock:/var/run/docker.sock

Command:

-v /var/run/docker.sock:/var/run/docker.sock
@MasterChiefv1
Copy link

 - /var/run/docker.sock:/var/run/docker.sock

Works.. thanks

@wereii
Copy link

wereii commented Sep 10, 2022

Does kuma write into the docker socket?

If not it should be made readonly: /var/run/docker.sock:/var/run/docker.sock:ro .

Otherwise, if kuma is compromised the attacker will have free access to docker (which is mostly running as root, so also to the whole host machine).

@MasterChiefv1
Copy link

Does kuma write into the docker socket?

If not it should be made readonly: /var/run/docker.sock:/var/run/docker.sock:ro .

Otherwise, if kuma is compromised the attacker will have free access to docker (which is mostly running as root, so also to the whole host machine).

That is a very good point.
let's wait and see what te developer says.

@louislam
Copy link
Owner

Yes, it should be read only.

@Meulator
Copy link
Author

Hello all,

it works flawlessly adding /var/run/docker.sock:/var/run/docker.sock:ro as a volume in docker-compose file.

Thanks a lot!

@ThelloD
Copy link

ThelloD commented Sep 15, 2022

Does kuma write into the docker socket?

If not it should be made readonly: /var/run/docker.sock:/var/run/docker.sock:ro .

Otherwise, if kuma is compromised the attacker will have free access to docker (which is mostly running as root, so also to the whole host machine).

This is not correct and therefore a very risky advice as users may mistakenly think they are safe with this configuration.

The docker.sock file is not a regular file on the file system, but - as the name implies - a socket. Read only permissions do not prevent interacting with a socket, meaning even if the socket is exposed as read-only, the application (or an attacker in case of a successful compromise of the system) has full control over every other container and also the host system, since privilege escalation or container escapes are trivial!

Although it's a neat feature to have this option, I strongly advice against granting uptime Kumo access to the socket if the application is exposed to the internet.

@joerich
Copy link

joerich commented Oct 21, 2022

By default, a docker container is self-contained and cannot access your host. I think you need to bind the /var/run/docker.sock to your container.

docker-compose:

volumes:
   - /var/run/docker.sock:/var/run/docker.sock

Command:

-v /var/run/docker.sock:/var/run/docker.sock

I'm having the same issue, but this didn't work for me.

I'm running uptime-kuma on my Terramaster NAS at 10.0.0.40
i've used -v /var/run/docker.sock:/var/run/docker.sock when setting up uptime and still get the same error.
I have no clue.

@joerich

This comment has been minimized.

@wereii
Copy link

wereii commented Nov 28, 2022

Are you sure the path exists on that NAS ? ls /var/run/docker.sock

@joerich
Copy link

joerich commented Nov 28, 2022

yes it is. I have no clue what the problem is.

@wereii
Copy link

wereii commented Nov 28, 2022

Could be some other restriction like SELinux ?

@joerich
Copy link

joerich commented Nov 28, 2022

root@TNAS-43EE:/var/run# cat docker.sock
cat: can't open 'docker.sock': No such device or address

@joerich
Copy link

joerich commented Nov 28, 2022

wierd! I'm looking at the file!

@joerich
Copy link

joerich commented Nov 28, 2022

I don't understand, I have 4 other containers running on this NAS! I'm really confused!

@joerich
Copy link

joerich commented Nov 28, 2022

on my NAS I'm running Uptime-kuma in a container therefore I don't think it can see "/var/run/docker.sock"

I just installed the standard version of Uptime-Kuma NOT DOCKER on my LINUX machine and it works.

I just read that if I'm using the Docker version I needed to bind something to the container?!

If that's so can I do it in Portainer, and what do I bind to it? This? "/var/run/docker.sock"

@IrwenXYZ
Copy link

Hi just want to add on that adding docker.sock doesn't work for me either.

volumes:
    - ./data:/app/data
    - /var/run/docker.sock:/app/data/docker.sock

Troubleshooting steps taken

ls /var/run/docker.sock both on the host and the container shows that it exists.

However, cat docker.sock shows cat: /var/run/docker.sock: No such device or address.

After some investigation I realised that mounting /var/run/docker.sock into /app/data/docker.sock works. I'm not sure why, could be some permissions issue.

@joerich
Copy link

joerich commented Nov 28, 2022

how do I mount /var/run/docker.sock into /app/data/docker.sock
I'm using portainer.

@IrwenXYZ
Copy link

Never used portainer, so I wouldn't know how to help specifically.

docker-compose:

volumes:
   - /var/run/docker.sock:/app/data/docker.sock

Command:

-v /var/run/docker.sock:/app/data/docker.sock

Of course, when in Uptime Kuma, enter /app/data/docker.sock as the Docker Daemon.

@joerich
Copy link

joerich commented Nov 29, 2022

I don't know how to use docker-compose can you help me?

@IrwenXYZ
Copy link

How do you run uptime kuma?

By default, a docker container is self-contained and cannot access your host. I think you need to bind the /var/run/docker.sock to your container.
docker-compose:

volumes:
   - /var/run/docker.sock:/var/run/docker.sock

Command:

-v /var/run/docker.sock:/var/run/docker.sock

I'm having the same issue, but this didn't work for me.

I'm running uptime-kuma on my Terramaster NAS at 10.0.0.40 i've used -v /var/run/docker.sock:/var/run/docker.sock when setting up uptime and still get the same error. I have no clue.

based on this, I think you'll just have to change -v /var/run/docker.sock:/var/run/docker.sock to -v /var/run/docker.sock:/app/data/docker.sock

@ryck ryck mentioned this issue Dec 8, 2022
5 tasks
@diomaxer
Copy link

diomaxer commented Aug 4, 2023

Hi guys. As far as I understand kuma pings the container, but is it possible to send a GET/POST request to it?

@CommanderStorm
Copy link
Collaborator

is it possible to send a GET/POST request to it?

Yes, that is what the http monitor is for

@diomaxer
Copy link

diomaxer commented Aug 4, 2023

Yes, that is what the http monitor is for

Maybe you misunderstood me. I connected to a docker host using (Method 2) TCP - Bridge Mode from https://github.com/louislam/uptime-kuma/wiki/How-to-Monitor-Docker-Containers. I want to send requests to my containers through it. I want to close all ports on my server except docker hub and send requests through docker network.

@chakflying
Copy link
Collaborator

The docker API does not provide such a functionality. Also, I don't think this is very related to the original discussion.

@CommanderStorm
Copy link
Collaborator

Yes, that is what the http monitor is for

No, I did not misunderstand you (I think)
Set up networking to allow communication between the two containers and use the http monitor.

Now I fully agree that this is not the nicest way to go about this, and maybe we can improve in this department, but it works.

@flowcool
Copy link

flowcool commented Aug 25, 2023

Hello everyone,

If I may add:
I'm using docker proxy:
https://github.com/Tecnativa/docker-socket-proxy

And that's working quite well from my perspective: don't have to run the container under root and don't need to give and access as docker proxy is managing this in read only.

My docker compose for uptime kuma and docker proxy:

version: '3.3'

services:
  uptime-kuma:
    image: louislam/uptime-kuma:1
    container_name: uptime-kuma
    environment:
      TZ: ${TZ}
      PUID: ${PUID}
      PGID: ${PGID}
    labels:
      - "swag=enable"
      - "swag_auth=authelia"
    ports:
      - 3001:3001  # <Host Port>:<Container Port>
    volumes:
      - ${DOCKERCONFDIR}/uptime-kuma:/app/data
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        max-file: ${DOCKERLOGGING_MAXFILE}
        max-size: ${DOCKERLOGGING_MAXSIZE}
    networks:
        - frontend
        - backend

  dockerproxy:
    image: ghcr.io/tecnativa/docker-socket-proxy:latest
    container_name: dockerproxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    restart: unless-stopped
    healthcheck:
      test: wget --spider http://localhost:2375/version || exit 1
      interval: "29s"
      timeout: "5s"
      retries: 3
      start_period: "21s"
    environment:
      - CONTAINERS=1
      - POST=0
    networks:
      - backend
    
networks:
  frontend:
    external: true
    name: frontend
  backend:
    external: true
    name: backend

2023-08-25_13h16_47

@Mouxy
Copy link

Mouxy commented Sep 1, 2023

Hey @louislam

It would be good to add to the documentation an example for people who installed docker using SNAP in Ubuntu.

If you installed docker using snap, the location of the files is different: daemon.json is in /var/snap/docker/current/config/

sudo nano /var/snap/docker/current/config/daemon.json

And this is what the original file looks like

{
    "log-level":        "error",
    "storage-driver":   "overlay2"
}

Change it to

{
    "log-level":        "error",
    "storage-driver":   "overlay2",
    "hosts": ["unix:///var/run/docker.sock", "tcp://0.0.0.0:2375"]
}
  • Restart the service:
    sudo systemctl restart snap.docker.dockerd.service

  • Check if the service is running:
    sudo systemctl status snap.docker.dockerd.service

And the result should be usual but with the snap location

RustDesk - 09-01 at 22 18 35@2x

@chakflying
Copy link
Collaborator

Feel free to make a PR on the wiki repo.

@ChristopherJTrent
Copy link

ChristopherJTrent commented Dec 14, 2023

I'm having the same issue here.
This is the uptime-kuma section of my docker-compose.

version: "3.8"

services:
  uptime-kuma:
    container_name: uptime-kuma
    image: louislam/uptime-kuma:latest
    restart: always
    ports:
      - 3001:3001
    volumes:
      - ./data/uptime-kuma:/app/data
      - //var/run/docker.sock:/var/run/docker.sock

I'm getting that ENOENT error on a fully up to date install of Arch Linux (linux-lts kernel) with fully up to date software.
I have verified that /var/run/docker.sock exists, and tried both //var and /var, as well as mapping it to a different internal directory.
I also tried using /run/docker.sock, since /var/run is a link to /run on my system, that didn't work either.

Edit to add, here's the output from the container when I attach to it and run the socket test:

Error log

``` AxiosError: connect ENOENT /var/run/docker.sock at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) { address: '/var/run/docker.sock', syscall: 'connect', code: 'ENOENT', errno: -2, config: { transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, adapter: [Function: httpAdapter], transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, env: { FormData: [Function] }, validateStatus: [Function: validateStatus], headers: { Accept: '*/*', 'User-Agent': 'Uptime-Kuma/1.23.10' }, url: '/containers/json?all=true', socketPath: '/var/run/docker.sock', method: 'get', data: undefined }, request: Writable { _writableState: WritableState { objectMode: false, highWaterMark: 16384, finalCalled: false, needDrain: false, ending: false, ended: false, finished: false, destroyed: false, decodeStrings: true, defaultEncoding: 'utf8', length: 0, writing: false, corked: 0, sync: true, bufferProcessing: false, onwrite: [Function: bound onwrite], writecb: null, writelen: 0, afterWriteTickInfo: null, buffered: [], bufferedIndex: 0, allBuffers: true, allNoop: true, pendingcb: 0, constructed: true, prefinished: false, errorEmitted: false, emitClose: true, autoDestroy: true, errored: null, closed: false, closeEmitted: false, [Symbol(kOnFinished)]: [] }, _events: [Object: null prototype] { response: [Function: handleResponse], error: [Function: handleRequestError], socket: [Function: handleRequestSocket] }, _eventsCount: 3, _maxListeners: undefined, _options: { maxRedirects: 21, maxBodyLength: 10485760, protocol: 'http:', path: '/containers/json?all=true', method: 'GET', headers: [Object], agent: undefined, agents: [Object], auth: undefined, socketPath: '/var/run/docker.sock', nativeProtocols: [Object], hostname: '::1', pathname: '/containers/json', search: '?all=true' }, _ended: true, _ending: true, _redirectCount: 0, _redirects: [], _requestBodyLength: 0, _requestBodyBuffers: [], _onNativeResponse: [Function (anonymous)], _currentRequest: ClientRequest { _events: [Object: null prototype], _eventsCount: 7, _maxListeners: undefined, outputData: [], outputSize: 0, writable: true, destroyed: false, _last: true, chunkedEncoding: false, shouldKeepAlive: false, maxRequestsOnConnectionReached: false, _defaultKeepAlive: true, useChunkedEncodingByDefault: false, sendDate: false, _removedConnection: false, _removedContLen: false, _removedTE: false, strictContentLength: false, _contentLength: 0, _hasBody: true, _trailer: '', finished: true, _headerSent: true, _closed: false, socket: [Socket], _header: 'GET /containers/json?all=true HTTP/1.1\r\n' + 'Accept: */*\r\n' + 'User-Agent: Uptime-Kuma/1.23.10\r\n' + 'Host: [::1]\r\n' + 'Connection: close\r\n' + '\r\n', _keepAliveTimeout: 0, _onPendingData: [Function: nop], agent: [Agent], socketPath: '/var/run/docker.sock', method: 'GET', maxHeaderSize: undefined, insecureHTTPParser: undefined, joinDuplicateHeaders: undefined, path: '/containers/json?all=true', _ended: false, res: null, aborted: false, timeoutCb: null, upgradeOrConnect: false, parser: null, maxHeadersCount: null, reusedSocket: false, host: '::1', protocol: 'http:', _redirectable: [Circular *1], [Symbol(kCapture)]: false, [Symbol(kBytesWritten)]: 0, [Symbol(kNeedDrain)]: false, [Symbol(corked)]: 0, [Symbol(kOutHeaders)]: [Object: null prototype], [Symbol(errored)]: null, [Symbol(kHighWaterMark)]: 16384, [Symbol(kRejectNonStandardBodyWrites)]: false, [Symbol(kUniqueHeaders)]: null }, _currentUrl: 'http://[::1]/containers/json?all=true', [Symbol(kCapture)]: false } } ```

@chakflying
Copy link
Collaborator

Have you checked that the "file" does exist inside the container in the path that you specified?

@ChristopherJTrent
Copy link

Have you checked that the "file" does exist inside the container in the path that you specified?

It does, looks like (for some unknown reason) the software doesn't like the link that's being forwarded into the container.
I'm not 100% certain what it was that I changed that fixed the issue, but it is working now with the following changes:
set the path to the socket in the GUI as /run/docker.sock

This fix didn't work previously, it's one of the things I tried, but after multiple reboots of the container I guess it just got with the program 🤷

@nullpanda
Copy link

nullpanda commented Dec 26, 2023

Saw the most recent comment about using /run/docker.sock because the container appears to be mapping /var/run to /run. I am having the same issue where I've exposed docker.sock in the compose file, I've validated that it exists properly at both /var/run and /run, I go into the UI and add the path and click test and get

connect ENOENT /var/run/docker.sock

I am running Ubuntu 22.04 as my host.

Any advice is much appreciated, monitoring the containers seems great!

Edit:
tried the /app/data fix again and this time it worked...wild.

@Gill-Bates
Copy link

Gill-Bates commented Feb 26, 2024

This solved it in my case

My Environment:

$ uname -a
Linux sv2 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux

docker-compose.yml

volumes:
  - /var/run/docker.sock:/app/data/docker.sock

Important! :ro (ReadOnly) will not work

image

@Nebulosa-Cat
Copy link

I'm trying to configure those docker.sock setting, and I notice seems uptime-kuma only use GET method to get data from path /containers/json and /containers/*/json ?

I'm trying this caddy setting and it work:

(my.domain) {
        tls my.email {
                dns cloudflare tokenHere
        }
}
# Docker Sock Expose
my.domain {
  	@containersUrl {
    	method GET
    	path /containers/json
  	}

  	@containerStatsUrl {
    	method GET
    	path /containers/*/json
  	}

  	reverse_proxy @containersUrl 127.0.0.1:2375
	reverse_proxy @containerStatsUrl 127.0.0.1:2375
	import my.domain
}

And it seems work, it should safe enough? It should make anyone only can use specific method and path to access docker socket

@MrMetacomCode
Copy link

Yes, it should be read only.

Can the documentation here be updated with this read only option set?

@CommanderStorm
Copy link
Collaborator

Please refer to @ThelloD in #2061 (comment) which summed up pretty good why just doing this would likely mislead people

This is not correct and therefore a very risky advice as users may mistakenly think they are safe with this configuration.

The docker.sock-file is not a regular file on the file system, but - as the name implies - a socket. Read only permissions do not prevent interacting with a socket, meaning even if the socket is exposed as read-only, the application (or an attacker in case of a successful compromise of the system) has full control over every other container and also the host system, since privilege escalation or container escapes are trivial!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment