Wednesday 9 November 2016

Use the --foreground option with timeout command and docker exec - prevents hang

TLDR; 1. the timeout command is a very useful command in scripts to make sure they don't get stuck on a hanging command

TLDR; 2. Using the --foreground option in timeout command which calls docker exec or other commands might prevent the command hanging.

e.g. timeout --foreground 5 docker exec -i su - -c ' '


https://github.com/moby/moby/issues/28207



docker exec hang when run using timeout command (and docker v1.12.2) #28207

 Open
Gaoithe opened this issue on Nov 9, 2016 · 11 comments

Comments

@Gaoithe
 
 

Gaoithe commented on Nov 9, 2016  

There are a few issues open around docker exec hanging but this one seems quite specific to running docker exec under the timeout command and to latest docker version 1.12.2 . . .
Have scripts which run docker exec like this:
timeout 1m docker exec -i $id sh -c "$command"
They had been working ok. After upgrade to docker 1.12.2 the scripts hang on those docker exec commands.
Interestingly, if I run the command on command-line (bash shell from ssh) then no hangs are seen.
If the command is run from inside a #!/bin/bash script then the hang happens all the time.

BUG REPORT INFORMATION

Description
docker exec command hangs when run in a bash script using timeout command and
same command using timeout doesn't hang from command-line
Steps to reproduce the issue:
  1. make a script like this called ugh.sh and run it:
#!/bin/bash
timeout 1m docker exec -it $contid sh -c "ls"
  1. chmod 755 ugh.sh; ./ugh.sh
NOTE that the very same command run in shell doesn't hang.



LATER:


 

pixelb commented on Nov 14, 2016

Have you tried the --foreground option to timeout? If docker is behaving as a monitor itself then this should timeout all children and be the least invasive way to run docker
@Gaoithe
 
 
Author

Gaoithe commented on Nov 14, 2016

Thank you @pixelb. That works. Interesting! Using timeout --foreground 1m docker exec . . . etc the commands now do not hang (when running in bash script from bash shell which is when we saw hang before.). The hang was not and is not seen when the timeout and docker exec was called interactively from shell. I realise now (slightly more) the significance of that.
That looks like a good workaround for our scripts.


https://stackpointer.io/unix/unix-linux-run-command-with-timeout/500/

USAGE:

$ timeout 1s sleep 10
$ echo $?
124

$ timeout 10s sleep 1
$ echo $?
0

$ timeout --help
Usage: timeout [OPTION] DURATION COMMAND [ARG]...
  or:  timeout [OPTION]
Start COMMAND, and kill it if still running after DURATION.

Mandatory arguments to long options are mandatory for short options too.
      --preserve-status
                 exit with the same status as COMMAND, even when the
                   command times out
      --foreground
                 when not running timeout directly from a shell prompt,
                   allow COMMAND to read from the TTY and get TTY signals;
                   in this mode, children of COMMAND will not be timed out
  -k, --kill-after=DURATION
                 also send a KILL signal if COMMAND is still running
                   this long after the initial signal was sent
  -s, --signal=SIGNAL
                 specify the signal to be sent on timeout;
                   SIGNAL may be a name like 'HUP' or a number;
                   see 'kill -l' for a list of signals
      --help     display this help and exit
      --version  output version information and exit

DURATION is a floating point number with an optional suffix:
's' for seconds (the default), 'm' for minutes, 'h' for hours or 'd' for days.

If the command times out, and --preserve-status is not set, then exit with
status 124.  Otherwise, exit with the status of COMMAND.  If no signal
is specified, send the TERM signal upon timeout.  The TERM signal kills
any process that does not block or catch that signal.  It may be necessary
to use the KILL (9) signal, since this signal cannot be caught, in which
case the exit status is 128+9 rather than 124.

GNU coreutils online help:
For complete documentation, run: info coreutils 'timeout invocation'



No comments: