Skip to content

devmapper: ensure that UdevWait is called after calls to setCookie#33732

Merged
cpuguy83 merged 1 commit intomoby:masterfrom
nhorman:cookie-clean-up
Jun 19, 2017
Merged

devmapper: ensure that UdevWait is called after calls to setCookie#33732
cpuguy83 merged 1 commit intomoby:masterfrom
nhorman:cookie-clean-up

Conversation

@nhorman
Copy link

@nhorman nhorman commented Jun 19, 2017

Recent changes to devmapper broke the implicit requirement that UdevWait be
called after every call to task.setCookie. Failure to do so results in leaks of
semaphores in the LVM code, eventually leading to semaphore exhaustion.
Previously this was handled by calling UdevWait in a ubiquitous defer function,
but that had its own problems with ordering between execution of the wait and
other uses of the semaphore. So now we delay returning until after the UdevWait
completes, once we've set the cookie value.

Problem can be observed by running moby with an lvm2 storage backend for extended periods of time. When enough cookies have leaked, then they problem will present.

Signed-off-by: Neil Horman nhorman@tuxdriver.com

@nhorman
Copy link
Author

nhorman commented Jun 19, 2017

@rhvgoyal as requested, tagging you to review this

@rhvgoyal
Copy link
Contributor

@nhorman How about going back to defer UdevWait(). That seems to be much simpler code.

IIRC, UdevWait() inline was redundant once we started doing allocation from heap?

Recent changes to devmapper broke the implicit requirement that UdevWait be
called after every call to task.setCookie.  Failure to do so results in leaks of
semaphores in the LVM code, eventually leading to semaphore exhaustion.
Previously this was handled by calling UdevWait in a ubiquitous defer function.
While there was initially some concern with deferring the UdevWait function
would cause some amount of race possibiliy, the fact that we never return the
cookie value or any value used to find it, makes that possibility seem unlikely,
so lets go back to that method

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
@rhvgoyal
Copy link
Contributor

cc @cpuguy83 @thaJeztah

@rhvgoyal
Copy link
Contributor

LGTM

@rhvgoyal
Copy link
Contributor

I did basic testing and it seems to fix cookies leaking issue when a busy device is being removed.

Copy link
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cpuguy83 cpuguy83 merged commit 7e3d0a5 into moby:master Jun 19, 2017
@thaJeztah
Copy link
Member

@rhvgoyal @nhorman IIUC, this is the PR that introduced this issue? #33119

@rhvgoyal
Copy link
Contributor

@thaJeztah Right. Last patch in that patch series introduced the issue.

@fitz123
Copy link

fitz123 commented Oct 4, 2017

Thank you! Finally it's released! Horaay

@leecade
Copy link

leecade commented Nov 21, 2017

cool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants