reptyr: Attach a running process to a new terminal

Over the last week, I’ve written a nifty tool that I call reptyr. reptyr is a utility for taking an existing running program and attaching it to a new terminal. Started a long-running process over ssh, but have to leave and don’t want to interrupt it? Just start a screen, use reptyr to grab it, and then kill the ssh session and head on home.

You can grab the source, or read on for some more details.

There’s a shell script called screenify that’s been going around the internet for nigh on 10 years now that is supposed to use gdb to accomplish the same thing. There’s also a project called retty that tries to do the same thing, in C using ptrace() directly.

The difference between those programs and reptyr is that reptyr works much, much, better.

If you attach a less using screenify or retty, it will still take input from the old terminal. If you attach an ncurses program, and resize the window, the program probably won’t resize correctly. ^C and ^Z will still be processed on the old terminal — typing them in the new terminal won’t do anything useful.

reptyr fixes all of these problems and more, and is the only such tool I know of that does so. I’ve never seen a program that doesn’t behave noticeably incorrectly after attaching with retty or screenify, whereas with reptyr most programs I have tried work flawlessly.

How does it work?

reptyr works in the same basic way as screenify and retty — it attaches to the target process using the ptrace API, opens the new terminal, and dup2s it over the old file descriptors. It also copies the termios settings from the old terminal to the new terminal.

The main thing that reptyr does that no one else does is that it actually changes the controlling terminal of the process you are attaching. This is the detail that makes many things Just Work, including ^C and ^Z and window resizing.

Switching the target’s controlling terminal is not easy and involves a fair bit of trickery with ptrace and Linux’s terminal APIs. I will probably do another blog post some time about the dirty details of how I make this work, but for now you can check out attach.c if you really want to know.

reptyr still has a number of limitations — it doesn’t generally work, for example, if the target process has any children. I know how to fix most of these problems, though, so expect it to get better with time. Please let me know if you find it useful!

Appendix

(Edited to add:) Nothing is really new. A commenter on reddit pointed out that injcode and neercs both accomplish the same thing, even using the same trick to change the CTTY. Ah well, I had run writing it anyways, and apparently I wasn’t the only one who didn’t know about the existing alternatives. neercs is a full screen replacement, though, and I think that reptyr should be more robust than injcode — I use a different techique for ptrace-hijacking, for example — and so hopefully this tool still has a niche as a more robust standalone utility. Certainly, judging from the amount of enthusiasm I’ve seen for this tool, this still isn’t a problem that is solved to the average user’s satisfaction.

Leave a comment | Trackback
Jan 21st, 2011
Tags: , , , ,
  1. jype
    Reply | Quote
    Jan 22nd, 2011 at 06:25 | #1

    I can’t imagine how anyone regularly working with linux machines could find this anything other than an extremely welcome addition to the toolbox. Thanks!

  2. RS
    Reply | Quote
    Jan 22nd, 2011 at 10:46 | #2

    Love it!

  3. Scott Conner
    Reply | Quote
    Jan 22nd, 2011 at 11:39 | #3

    Incredible analysis of the existing tools out there. Good job man. Keep it up.

  4. ward
    Reply | Quote
    Jan 22nd, 2011 at 18:20 | #4

    Sounds fantastic, but it does not work for me:

    ./reptyr 3838
[+] Allocated scratch page: b789f000
[+] Looking up fds for tty in child.
[+] Resolved child tty: /dev/pts/164
[+] Found an alias for the tty: 0
[+] Found an alias for the tty: 1
[+] Found an alias for the tty: 2
Unable to attach to pid 3838: Permission denied

    This is on Ubuntu Maverick. Trying to attach a vim session.

  5. nelhage
    Reply | Quote
    Jan 22nd, 2011 at 18:55 | #5

    @ward Are you able to run it under strace and send me the output? That would be the easiest way for me to try to figure out what’s going wrong.

  6. px
    Reply | Quote
    Jan 22nd, 2011 at 19:10 | #6

    Holy god man, nice work. When I connected to a vim process the up/down arrows didn’t seem to function properly, and it would be nice if it cleanly returned the other shell.

    Also starting in 10.10 ubuntu has ptrace disabled except for child processes and certain exceptions negotiated with kees :-) To disable it, you have to

    echo 0 > /proc/sys/kernel/yama/ptrace_scope

    But I wouldn’t recommend it for long term, it’s like that for some very good reasons.

  7. Bob
    Reply | Quote
    Jan 22nd, 2011 at 19:47 | #7

    There are at least two programs that do this already.

    injcode:

    http://blog.habets.pp.se/2009/03/Moving-a-process-to-another-terminal

    And neercs:

    http://caca.zoy.org/wiki/neercs

  8. nm
    Reply | Quote
    Jan 22nd, 2011 at 20:14 | #8

    Very nice, a fairly old hack but nice to have something without the usual problems.

    You may be interested to know there is a program called neercs which also does something similar although I haven’t tried it and don’t know what problems it has.

    I wish someone would come up with a clean way to do this and I’d be all over it for tmux. I think it’d probably need kernel support though to be worth it; a kernel API would make it easier to port as well.

  9. drhodes
    Reply | Quote
    Jan 22nd, 2011 at 23:58 | #9

    Works here on x86 Ubuntu 10.4 LTS; tested flawlessly with emacs and watch.

  10. MyrddinE
    Reply | Quote
    Jan 23rd, 2011 at 00:15 | #10

    If I didn’t have a permanent screen up on every box I use, this would be more useful. But the moment I connect to a box for the first time, I’m running screen or screen -x…

    If I didn’t do that, this would be awesome. :-)

  11. Greyed
    Reply | Quote
    Jan 23rd, 2011 at 03:10 | #11

    Exactly. This was answered literally decades ago by screen.

  12. Ceren
    Reply | Quote
    Jan 23rd, 2011 at 04:24 | #12

    @Greyed

    no, this is for when you have a process running that wasn’t started in screen, I think.

    Haven’t you ever started a process and thought “shiiiiit, I really should have run that in a screen session…”

  13. Rukke
    Reply | Quote
    Jan 23rd, 2011 at 06:29 | #13

    Love it!

  14. awe_cz
    Reply | Quote
    Jan 23rd, 2011 at 07:49 | #14

    Yes, I do find it really useful, thanks!

  15. nelhage
    Reply | Quote
    Jan 23rd, 2011 at 12:56 | #15
    px :

    When I connected to a vim process the up/down arrows didn’t seem to function properly, and it would be nice if it cleanly returned the other shell.

    The arrow keys is a known problem — the issue is that the arrow keys are configured by state that is kept by the terminal emulator, not by the in-kernel terminal system, and so doesn’t get copied by repytr. If you background vim (via ^Z in the new terminal), and then continue it (via either bg in the old terminal, or kill -CONT VIM-PID), vim should reinitialize the terminal and restore the state correctly.

    Alternately, it’s vim — what do you need arrow keys for? ;)

  16. LaC
    Reply | Quote
    Jan 23rd, 2011 at 13:16 | #16

    Doesn’t reptyr do the same thing as this? Which, in turn, did something that this is supposed to do as well. I guess someone will find a use for all these wheels…

  17. Jack
    Reply | Quote
    Jan 24th, 2011 at 12:19 | #17

    @Ceren @Greyed @nelhage

    Nelhage: You have effectively succeeded in not explaining what it is you are doing. You mention screen, you write ten not inconsequential paragraphs, make a stab at a usage scenario, but absolutely fail to give a non-relative explanation of what you are doing.

    You say “like X, akin to Y”.

    I use screen, I know that ^s annoys me, as does ^a being taken over. You say start a screen, and use reptyr – use it for what?

    So, It starts in a screen, and reptyr is for attaching to that process onto a normal terminal (and not using screen again to connect to it?)

    If that is what it does THEN SAY THAT. Why it is that you haven’t said it? I’ve pastebin’d your original comment for reference, because it just irks me that people are unable to grasps, understand, and coordinate language in a clear-minded way.

  18. cruz
    Reply | Quote
    Jan 24th, 2011 at 14:19 | #18

    Now this is fun code, and when you need this tool you really need it. I hope to find a mature version of this tool on every linux box in a couple years. Great work.

  19. danath
    Reply | Quote
    Feb 17th, 2011 at 05:32 | #19

    Many thanks for making this tool.

    However I failed to run it on my ARM device with the error message:

    “Target is not connected to a terminal.”

    I checked the /proc and found the stdout was /dev/null.

    So it means the tool does not handle this case?

  20. nelhage
    Reply | Quote
    Feb 20th, 2011 at 04:56 | #20

    @danath The tool was originally written to attach a process that was already connected to a terminal, to a different terminal. I’ve just pushed a new branch to github — attach-fds — that supports a -s option that causes it to attach and grab fds 0-2 even if the target is not already attached to a terminal. Let me know if that makes it work in your case.

  21. danath
    Reply | Quote
    Feb 21st, 2011 at 05:44 | #21

    @nelhage It works well! Many thanks to you.

    Looking forward to having this powerful tool in the standard coreutils or busybox release.

    BTW, my machine is Linux 2.6.27 running on ARM 926.

  22. Andrew
    Reply | Quote
    Apr 28th, 2011 at 04:08 | #22

    A Naive comment here, as I don’t know all that much about the structures involved:

    Maybe what we want isn’t so much a way to move a process to a new tty, as to move an existing tty under the control of screen?

    I understand that solving that problem would be quite a different proposition, but if it’s doable it might be more general and more robust?

  23. AndyE
    Reply | Quote
    Apr 28th, 2011 at 08:18 | #23

    Yeah, people worked on something similar, but who would deny you your fun. I hope you continue to work on it and provide updates. It IS extremely useful when you need it.

    Oh, and Jack, you just don’t have a clue. This article was well written and can be perfectly understood. The title of the page pretty much tells you everything you need to know. The only thing lacking here is your comprehension ability and your humility

  24. Christoph Anton Mitterer
    Reply | Quote
    Jul 16th, 2011 at 14:51 | #24

    Hey.

    Great work. :-)

    “I know how to fix most of these problems” … is this already done? I’d love to see all that in code ;)

    Cheers, Chris.

  25. KrusjMe
    Reply | Quote
    Jul 25th, 2011 at 07:49 | #25

    Can it be possible that reptyr doesn’t work on PPC-based linux? My Gentoo64 PPC says that this architecture is not supported. (when trying to compile)

  26. nelhage
    Reply | Quote
    Jul 25th, 2011 at 14:30 | #26

    @Christoph Anton Mitterer I’ve fixed a number of issues since this post went up, including releasing the old terminal so it can be used again, and implementing support for attaching compat-mode 32-bit processes on amd64. I haven’t fixed attaching things with children; Maybe I’ll attempt to tackle it one of these days.

  27. nelhage
    Reply | Quote
    Jul 25th, 2011 at 14:40 | #27

    @KrusjMe Correct, reptyr currently only supports x86 and ARM Linux machines, as noted in the README. Adding support for PPC probably wouldn’t be too difficult, but I don’t have the time or the inclination to do it at the moment, although I would probably accept patches.

  28. KrusjMe
    Reply | Quote
    Jul 27th, 2011 at 12:32 | #28

    @nelhage: Thank you for your quick response. Ah, the README! :) I tried to install it on Gentoo PPC64 using Portage. (the default package-manager of Gentoo) Portage just said something about the architecture. Thanks again!

  29. mzeta
    Reply | Quote
    Sep 12th, 2011 at 23:27 | #29

    same problem than @ward here

    root@haku:~/reptyr# ./reptyr 30386
[+] Allocated scratch page: b778e000
[+] Looking up fds for tty in child.
[+] Resolved child tty: 8801
[+] Found an alias for the tty: 0
[+] Found an alias for the tty: 1
[+] Found an alias for the tty: 2
[+] Found an alias for the tty: 3
[-] Unable to open the tty in the child.
Unable to attach to pid 30386: Permission denied

    this is debian sid. Also I have no /proc/sys/kernel/yama folder nor /etc/sysctl.d/10-ptrace.conf file so I can’t go on

    echo 0 > /proc/sys/kernel/yama/ptrace_scope

    If applicable, I can email you straced output for the call

    the tool however sounds to be a nice hack.

  30. nelhage
    Reply | Quote
    Sep 13th, 2011 at 11:54 | #30

    @mzeta Hm, I think that is actually a slightly different error from ward’s. If you could email me the strace output, that would be helpful.

  31. Casper
    Reply | Quote
    Oct 14th, 2011 at 08:27 | #31

    Works great! (Debian Squeeze)

  32. Lachlan
    Reply | Quote
    Dec 21st, 2011 at 01:56 | #32

    You are a fricking GOD!!!

    I needed this realllllllllll bad today. :)

  33. vMeson
    Reply | Quote
    Dec 28th, 2011 at 14:31 | #33

    “I had run writing it anyways” typolice: s/had run/had fun/

    Adding ppc and mips support is now on my to do list. It’s a long list so don’t get your hopes up. :-)

  34. Jean
    Reply | Quote
    Jan 25th, 2012 at 13:43 | #34

    This is incredible stuff ! Thank you Mister nelhage

  35. Alexander Kriegisch
    Reply | Quote
    Feb 1st, 2012 at 08:11 | #35

    I have the same problem as mzeta. Did he send you the strace output and did you come up with an idea how to solve this? Me running coLinux makes it next to impossible to just switch to another (yama-enabled) kernel, but I do use Ubuntu 11.10 which seems to silently rely on the fact that there is just such a kernel running underneath it. In my case it isn’t.

    $ uname -a Linux andLinux 2.6.33.7-co-0.7.10-r1588 #1 PREEMPT Mon Aug 8 04:13:31 UTC 2011 i686 athlon i386 GNU/Linux

    $ cat /etc/motd | head -n 1 Welcome to Ubuntu 11.10 (GNU/Linux 2.6.33.7-co-0.7.10-r1588 i686)

  36. Johann Pascher
    Reply | Quote
    Feb 3rd, 2012 at 13:48 | #36

    Hi Alexander Kriegisch (kriegaex),

    I don’t have any problem with it on Ubuntu 12.04 (speedlinux) Did test with this example: http://monkeypatch.me/blog/move-a-running-process-to-a-new-screen-shell.html

    uname -a Linux speedLinux 2.6.33.7-co-0.7.9-r1581 #1 PREEMPT Sat Apr 9 21:08:08 UTC 2011 i686 athlon i386 GNU/Linux cat /etc/motd | head -n 1 Welcome to Ubuntu precise (development branch) (GNU/Linux 2.6.33.7-co-0.7.9-r1581 i686)

    Best regards, Johann Pascher

  37. Kevin
    Reply | Quote
    Mar 8th, 2012 at 05:27 | #37

    Brilliant. Thanks a lot!

  38. Kot
    Reply | Quote
    Jun 7th, 2012 at 11:23 | #38

    That;s a really cool tool man, why don;t you try to put that in the Fedora repository or some other linux repository.

  39. nelhage
    Reply | Quote
    Jun 7th, 2012 at 11:27 | #39

    @Kot It’s actually been in (at least) Fedora, Debian, and Ubuntu for a while now. http://packages.debian.org/search?keywords=reptyr https://admin.fedoraproject.org/pkgdb/acls/name/reptyr

  40. James
    Reply | Quote
    Jun 25th, 2012 at 05:28 | #40

    Please add support for reptyr’ing a process and all of its children. I had a use for this today, but luckily caught the note about children processes before I messed up my 6hr build. Even though Neercs and the like exist, your tool is light-weight and works without having to install a whole client/server framework. It’s a great swiss-army knife style utility that makes Linux so great!

  41. Mike Grant
    Reply | Quote
    Sep 1st, 2012 at 11:01 | #41

    Support for grabbing the child processes would be very nice (e.g. for grabbing a running bash script) – I could see using that quite often.

    Presumably the basic idea would be to simultaneously suspend the process and all its children (any way to use process groups?), individually grab each one, then resume them all?

  42. nelhage
    Reply | Quote
    Sep 3rd, 2012 at 15:05 | #42

    @Mike Grant Yeah, that is basically right. There are some complications around dealing with zombie children, but I think in most cases it should be pretty straightforward.

  43. Colum Paget
    Reply | Quote
    Feb 7th, 2013 at 14:01 | #43

    Thanks for creating this, it’s great! I can move mpg123 around from terminal to termina!

  44. Vinod
    Reply | Quote
    Feb 28th, 2013 at 19:25 | #44

    Does anyone have ptrace files for MIP architecture?

    Thanks in advance.

  45. Vinod
    Reply | Quote
    Feb 28th, 2013 at 19:27 | #45

    @Vinod Typo: MIPS architecture only

Leave a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
RSS for comments on this post