Docker Container无响应/挂起问题

其中一个容器被挂起，并在syslogs中出现以下错误消息而无响应，任何帮助指出问题将不胜感激，

sys日志错误：

Jul 20 12:58:26 B2BTestServer dockerd[1140]: time=“2017-07-20T12:58:26.879111489Z” level=error msg=“containerd: deleting container” error=“exit status 1: “container 5f090b50b3dd7840dd296bd3eede2c4db171b5787944317f0a0e52d71c368361 does not exist\none or more of the container deletions failed\n”” Jul 20 13:30:01 B2BTestServer CRON[42898]: (xxxx) CMD (docker exec xxApp_worker_1 ./artisan omni:key b2b_fwad login >/dev/null 2>&1) Jul 20 13:30:01 B2BTestServer CRON[42899]: (xxxx) CMD (docker exec xxApp_app_1 ./artisan omni:key b2b_fwad login >/dev/null 2>&1) Jul 20 13:30:01 B2BTestServer CRON[42901]: (xxxx) CMD (docker exec xxApp_worker_1 ./artisan omni:key b2b_yww login >/dev/null 2>&1) Jul 20 13:30:01 B2BTestServer CRON[42900]: (xxxx) CMD (docker exec xxApp_app_1 ./artisan omni:key b2b_yww login >/dev/null 2>&1) Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.382668472Z” level=error msg="Error setting up exec command in container xxApp_app_1: Container f743904a3558b5475df8e574e9b27a5ca8b2c7256da59c356f4da1a9bda72e4b is not running" Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.382714110Z” level=error msg="Handler for POST /v1.25/containers/xxApp_app_1/exec returned error: Container f743904a3558b5475df8e574e9b27a5ca8b2c7256da59c356f4da1a9bda72e4b is not running" Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.392014905Z” level=error msg="Error setting up exec command in container xxApp_app_1: Container f743904a3558b5475df8e574e9b27a5ca8b2c7256da59c356f4da1a9bda72e4b is not running" Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.392237089Z” level=error msg="Handler for POST /v1.25/containers/xxApp_app_1/exec returned error: Container f743904a3558b5475df8e574e9b27a5ca8b2c7256da59c356f4da1a9bda72e4b is not running" Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.429292849Z” level=error msg="Error running exec in container: rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused “process_linux.go:83: executing setns process caused \“exit status 15\””\n" Jul 20 13:30:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:01.463663587Z” level=error msg=“Error running exec in container: rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused “process_linux.go:83: executing setns process caused \“exit status 15\””\n” Jul 20 13:30:36 B2BTestServer kernel: [262136.359656] aufs au_opts_verify:1597:dockerd[2634]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer kernel: [262136.583654] aufs au_opts_verify:1597:dockerd[2634]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.120768757Z” level=error msg=“containerd: deleting container” error="exit status 1: “container f80c3f7d41bcbb2133036c795068ea2200b8043bce1a043f5fb5e09d09c2f720 does not exist\none or more of the container deletions failed\n”" Jul 20 13:30:37 B2BTestServer kernel: [262136.651251] br-e80847589bfa: port 2(vethe12115e) entered disabled state Jul 20 13:30:37 B2BTestServer kernel: [262136.668608] vethebfd8d9: renamed from eth0 Jul 20 13:30:37 B2BTestServer kernel: [262136.716289] aufs au_opts_verify:1597:dockerd[2634]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer kernel: [262136.738679] br-e80847589bfa: port 2(vethe12115e) entered disabled state Jul 20 13:30:37 B2BTestServer kernel: [262136.753388] device vethe12115e left promiscuous mode Jul 20 13:30:37 B2BTestServer kernel: [262136.762750] br-e80847589bfa: port 2(vethe12115e) entered disabled state Jul 20 13:30:37 B2BTestServer kernel: [262136.804865] device vethdb0a6ac entered promiscuous mode Jul 20 13:30:37 B2BTestServer kernel: [262136.814982] IPv6: ADDRCONF(NETDEV_UP): vethdb0a6ac: link is not ready Jul 20 13:30:37 B2BTestServer kernel: [262136.827291] br-e80847589bfa: port 1(vethdb0a6ac) entered forwarding state Jul 20 13:30:37 B2BTestServer kernel: [262136.840256] br-e80847589bfa: port 1(vethdb0a6ac) entered forwarding state Jul 20 13:30:37 B2BTestServer systemd-udevd[43041]: Could not generate persistent MAC address for vethdb0a6ac: No such file or directory Jul 20 13:30:37 B2BTestServer systemd-udevd[43040]: Could not generate persistent MAC address for vethf7a98cb: No such file or directory Jul 20 13:30:37 B2BTestServer kernel: [262137.026630] aufs au_opts_verify:1597:dockerd[1209]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.579534632Z” level=warning msg="Your kernel does not support swap memory limit." Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.579622593Z” level=warning msg="Your kernel does not support cgroup rt period" Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.579632100Z” level=warning msg="Your kernel does not support cgroup rt runtime" Jul 20 13:30:37 B2BTestServer kernel: [262137.101169] aufs au_opts_verify:1597:dockerd[1209]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37Z” level=info msg="Firewalld running: false" Jul 20 13:30:37 B2BTestServer kernel: [262137.207004] aufs au_opts_verify:1597:dockerd[25633]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:37 B2BTestServer kernel: [262137.248708] eth0: renamed from vethf7a98cb Jul 20 13:30:37 B2BTestServer systemd-udevd[43171]: Could not generate persistent MAC address for veth91e8259: No such file or directory Jul 20 13:30:37 B2BTestServer systemd-udevd[43172]: Could not generate persistent MAC address for veth3fff19d: No such file or directory Jul 20 13:30:37 B2BTestServer kernel: [262137.273642] device veth3fff19d entered promiscuous mode Jul 20 13:30:37 B2BTestServer kernel: [262137.287783] IPv6: ADDRCONF(NETDEV_UP): veth3fff19d: link is not ready Jul 20 13:30:37 B2BTestServer kernel: [262137.307604] br-e80847589bfa: port 2(veth3fff19d) entered forwarding state Jul 20 13:30:37 B2BTestServer kernel: [262137.320987] br-e80847589bfa: port 2(veth3fff19d) entered forwarding state Jul 20 13:30:37 B2BTestServer kernel: [262137.335059] IPv6: ADDRCONF(NETDEV_CHANGE): vethdb0a6ac: link becomes ready Jul 20 13:30:37 B2BTestServer kernel: [262137.349736] br-e80847589bfa: port 2(veth3fff19d) entered disabled state Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.883496197Z” level=warning msg="Your kernel does not support swap memory limit." Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.883793205Z” level=warning msg="Your kernel does not support cgroup rt period" Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37.883937606Z” level=warning msg="Your kernel does not support cgroup rt runtime" Jul 20 13:30:37 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:37Z” level=info msg="Firewalld running: false" Jul 20 13:30:38 B2BTestServer kernel: [262137.524751] eth0: renamed from veth91e8259 Jul 20 13:30:38 B2BTestServer kernel: [262137.544382] IPv6: ADDRCONF(NETDEV_CHANGE): veth3fff19d: link becomes ready Jul 20 13:30:38 B2BTestServer kernel: [262137.556891] br-e80847589bfa: port 2(veth3fff19d) entered forwarding state Jul 20 13:30:38 B2BTestServer kernel: [262137.569301] br-e80847589bfa: port 2(veth3fff19d) entered forwarding state Jul 20 13:30:38 B2BTestServer kernel: [262137.704615] aufs au_opts_verify:1597:dockerd[3229]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:38 B2BTestServer kernel: [262137.803344] aufs au_opts_verify:1597:dockerd[3229]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:38 B2BTestServer kernel: [262137.902578] aufs au_opts_verify:1597:dockerd[1209]: dirperm1 breaks the protection by the permission bits on the lower branch Jul 20 13:30:38 B2BTestServer kernel: [262137.926880] device veth4ff9aca entered promiscuous mode Jul 20 13:30:38 B2BTestServer systemd-udevd[43302]: Could not generate persistent MAC address for veth4ff9aca: No such file or directory Jul 20 13:30:38 B2BTestServer kernel: [262137.938135] IPv6: ADDRCONF(NETDEV_UP): veth4ff9aca: link is not ready Jul 20 13:30:38 B2BTestServer systemd-udevd[43301]: Could not generate persistent MAC address for vetha4d818b: No such file or directory Jul 20 13:30:38 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:38.495129396Z” level=warning msg="Your kernel does not support swap memory limit." Jul 20 13:30:38 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:38.495215954Z” level=warning msg="Your kernel does not support cgroup rt period" Jul 20 13:30:38 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:38.495226161Z” level=warning msg="Your kernel does not support cgroup rt runtime" Jul 20 13:30:38 B2BTestServer dockerd[1140]: time=“2017-07-20T13:30:38Z” level=info msg="Firewalld running: false" Jul 20 13:30:38 B2BTestServer kernel: [262138.116861] eth0: renamed from vetha4d818b Jul 20 13:30:38 B2BTestServer kernel: [262138.136455] IPv6: ADDRCONF(NETDEV_CHANGE): veth4ff9aca: link becomes ready Jul 20 13:30:38 B2BTestServer kernel: [262138.149540] br-e80847589bfa: port 4(veth4ff9aca) entered forwarding state Jul 20 13:30:38 B2BTestServer kernel: [262138.162430] br-e80847589bfa: port 4(veth4ff9aca) entered forwarding state Jul 20 13:35:31 B2BTestServer dockerd[1140]: time=“2017-07-20T13:35:31.489837293Z” level=error msg="Handler for POST /v1.25/containers/xxApp_app_1/start returned error: Container already started" Jul 20 13:36:05 B2BTestServer dockerd[1140]: time=“2017-07-20T13:36:05.230362142Z” level=error msg=“Handler for POST /v1.25/containers/xxApp_worker_1/start returned error: Container already started” Jul 20 13:50:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:50:01.606622410Z” level=error msg="Error running exec in container: rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused “process_linux.go:83: executing setns process caused \“exit status 15\””\n" Jul 20 13:50:01 B2BTestServer dockerd[1140]: time=“2017-07-20T13:50:01.687809674Z” level=error msg=“Error running exec in container: rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused “process_linux.go:83: executing setns process caused \“exit status 15\””\n” 

以下是码头版本信息：

 sudo docker info Containers: 8 Running: 3 Paused: 0 Stopped: 5 Images: 14 Server Version: 1.13.0 Storage Driver: aufs Root Dir: /var/lib/docker/aufs Backing Filesystem: extfs Dirs: 126 Dirperm1 Supported: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e init version: 949e6fa Security Options: apparmor seccomp Profile: default Kernel Version: 4.4.0-83-generic Operating System: Ubuntu 16.04.1 LTS OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 6.804 GiB Name: TestServer Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ WARNING: No swap limit support Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false