Tutorial :Multiple glibc libraries on a single host



Question:

Multiple glibc libraries on a single host

My linux (SLES-8) server currently has glibc-2.2.5-235, but I have a program which won't work on this version and requires glibc-2.3.3.

Is it possible to have multiple glibcs installed on the same host?

This is the error I get when I run my program on the old glibc:

./myapp: /lib/i686/libc.so.6: version `GLIBC_2.3' not found (required by ./myapp)  ./myapp: /lib/i686/libpthread.so.0: version `GLIBC_2.3.2' not found (required by ./myapp)  ./myapp: /lib/i686/libc.so.6: version `GLIBC_2.3' not found (required by ./libxerces-c.so.27)  ./myapp: /lib/ld-linux.so.2: version `GLIBC_2.3' not found (required by ./libstdc++.so.6)  ./myapp: /lib/i686/libc.so.6: version `GLIBC_2.3' not found (required by ./libstdc++.so.6)  

So I created a new directory called newglibc and copied the following files in:

libpthread.so.0  libm.so.6  libc.so.6  ld-2.3.3.so  ld-linux.so.2 -> ld-2.3.3.so  

and

export LD_LIBRARY_PATH=newglibc:$LD_LIBRARY_PATH  

But I get an error:

./myapp: /lib/ld-linux.so.2: version `GLIBC_PRIVATE' not found (required by ./newglibc/libpthread.so.0)  ./myapp: /lib/ld-linux.so.2: version `GLIBC_2.3' not found (required by libstdc++.so.6)  ./myapp: /lib/ld-linux.so.2: version `GLIBC_PRIVATE' not found (required by ./newglibc/libm.so.6)  ./myapp: /lib/ld-linux.so.2: version `GLIBC_2.3' not found (required by ./newglibc/libc.so.6)  ./myapp: /lib/ld-linux.so.2: version `GLIBC_PRIVATE' not found (required by ./newglibc/libc.so.6)  

So it appears that they are still linking to /lib and not picking up from where I put them?

Thanks


Solution:1

It is very possible to have multiple versions of glibc on the same system (we do that every day).

However, you need to know that glibc consists of many pieces (200+ shared libraries) which all must match. One of the pieces is ld-linux.so.2, and it must match libc.so.6, or you'll see the errors you are seeing.

The absolute path to ld-linux.so.2 is hard-coded into the executable at link time, and can not be easily changed after the link is done.

To build an executable that will work with the new glibc, do this:

g++ main.o -o myapp ... \     -Wl,--rpath=/path/to/newglibc \     -Wl,--dynamic-linker=/path/to/newglibc/ld-linux.so.2  

The -rpath linker option will make the runtime loader search for libraries in /path/to/newglibc (so you wouldn't have to set LD_LIBRARY_PATH before running it), and the -dynamic-linker option will "bake" path to correct ld-linux.so.2 into the application.

If you can't relink the myapp application (e.g. because it is a third-party binary), not all is lost, but it gets trickier. One solution is to set a proper chroot environment for it. Another possibility is to use rtldi and a binary editor.


Solution:2

Use LD_PRELOAD: put your library somewhere out of the man lib directories and run:

LD_PRELOAD='mylibc.so anotherlib.so' program  

See: the Wikipedia article


Solution:3

This question is old, the other answers are old. "Employed Russian"s answer is very good and informative, but it only works if you have the source code. If you don't, the alternatives back then were very tricky. Fortunately nowadays we have a simple solution to this problem (as commented in one of his replies), using patchelf. All you have to do is:

$ ./patchelf --set-interpreter /path/to/newglibc/ld-linux.so.2 --set-rpath /path/to/newglibc/ myapp  

And after that, you can just execute your file:

$ ./myapp  

No need to chroot or manually edit binaries, thankfully. But remember to backup your binary before patching it, if you're not sure what you're doing, because it modifies your binary file. After you patch it, you can't restore the old path to interpreter/rpath. If it doesn't work, you'll have to keep patching it until you find the path that will actually work... Well, it doesn't have to be a trial-and-error process. For example, in OP's example, he needed GLIBC_2.3, so you can easily find which lib provides that version using strings:

$ strings /lib/i686/libc.so.6 | grep GLIBC_2.3  $ strings /path/to/newglib/libc.so.6 | grep GLIBC_2.3  

In theory, the first grep would come empty because the system libc doesn't have the version he wants, and the 2nd one should output GLIBC_2.3 because it has the version myapp is using, so we know we can patchelf our binary using that path.

When you try to run a binary in linux, the binary tries to load the linker, then the libraries, and they should all be in the path and/or in the right place. If your problem is with the linker and you want to find out which path your binary is looking for, you can find out with this command:

$ readelf -l myapp | grep interpreter    [Requesting program interpreter: /lib/ld-linux.so.2]                                                                                                                                                                                     

If your problem is with the libs, commands that will give you the libs being used are:

$ readelf -d myapp | grep Shared  $ ldd myapp   

This will list the libs that your binary needs, but you probably already know the problematic ones, since they are already yielding errors as in OP's case.

"patchelf" works for many different problems that you may encounter while trying to run a program, related to these 2 problems. For example, if you get: ELF file OS ABI invalid, it may be fixed by setting a new loader (the --set-interpreter part of the command) as I explain here. Another example is for the problem of getting No such file or directory when you run a file that is there and executable, as exemplified here. In that particular case, OP was missing a link to the loader, but maybe in your case you don't have root access and can't create the link. Setting a new interpreter would solve your problem.

Thanks Employed Russian and Michael Pankov for the insight and solution!


Solution:4

Can you consider using Nix http://nixos.org/nix/ ?

Nix supports multi-user package management: multiple users can share a common Nix store securely, don’t need to have root privileges to install software, and can install and use different versions of a package.


Solution:5

First of all, the most important dependency of each dynamically linked program is the linker. All so libraries must match the version of the linker.

Let's take simple exaple: I have the newset ubuntu system where I run some program (in my case it is D compiler - ldc2). I'd like to run it on the old CentOS, but because of the older glibc library it is impossible. I got

ldc2-1.5.0-linux-x86_64/bin/ldc2: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by ldc2-1.5.0-linux-x86_64/bin/ldc2)  ldc2-1.5.0-linux-x86_64/bin/ldc2: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ldc2-1.5.0-linux-x86_64/bin/ldc2)  

I have to copy all dependencies from ubuntu to centos. The proper method is following:

First, let's check all dependencies:

ldd ldc2-1.5.0-linux-x86_64/bin/ldc2       linux-vdso.so.1 =>  (0x00007ffebad3f000)      librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f965f597000)      libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f965f378000)      libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f965f15b000)      libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f965ef57000)      libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f965ec01000)      libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f965e9ea000)      libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f965e60a000)      /lib64/ld-linux-x86-64.so.2 (0x00007f965f79f000)  

linux-vdso.so.1 is not a real library and we don't have to care about it.

/lib64/ld-linux-x86-64.so.2 is the linker, which is used by the linux do link the executable with all dynamic libraries.

Rest of the files are real libraries and all of them together with the linker must be copied somewhere in the centos.

Let's assume all the libraries and linker are in "/mylibs" directory.

ld-linux-x86-64.so.2 - as I've already said - is the linker. It's not dynamic library but static executable. You can run it and see that it even have some parameters, eg --library-path (I'll return to it).

On the linux, dynamically linked program may be lunched just by its name, eg

/bin/ldc2  

Linux loads such program into RAM, and checks which linker is set for it. Usually, on 64-bit system, it is /lib64/ld-linux-x86-64.so.2 (in your filesystem it is symbolic link to the real executable). Then linux runs the linker and it loads dynamic libraries.

You can also change this a little and do such trick:

/mylibs/ld-linux-x86-64.so.2 /bin/ldc2  

It is the method for forcing the linux to use specific linker.

And now we can return to the mentioned earlier parameter --library-path

/mylibs/ld-linux-x86-64.so.2 --library-path /mylibs /bin/ldc2  

It will run ldc2 and load dynamic libraries from /mylibs.

This is the method to call the executable with choosen (not system default) libraries.


Solution:6

If you look closely at the second output you can see that the new location for the libraries is used. Maybe there are still missing libraries that are part of the glibc.

I also think that all the libraries used by your program should be compiled against that version of glibc. If you have access to the source code of the program, a fresh compilation appears to be the best solution.


Solution:7

"Employed Russian" is among the best answer, and I think all other suggested answer may not work. The reason is simply because when an application is first created, all its the APIs it needs are resolved at compile time. Using "ldd" u can see all the statically linked dependencies:

ldd /usr/lib/firefox/firefox      linux-vdso.so.1 =>  (0x00007ffd5c5f0000)      libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f727e708000)      libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f727e500000)      libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f727e1f8000)      libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f727def0000)      libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f727db28000)      /lib64/ld-linux-x86-64.so.2 (0x00007f727eb78000)      libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f727d910000)  

But at runtime, firefox will also load many other dynamic libraries, eg (for firefox) there are many "glib"-labelled libraries loaded (even though statically linked there are none):

 /usr/lib/x86_64-linux-gnu/libdbus-glib-1.so.2.2.2   /lib/x86_64-linux-gnu/libglib-2.0.so.0.4002.0   /usr/lib/x86_64-linux-gnu/libavahi-glib.so.1.0.2  

Manytimes, you can see names of one version being soft-linked into another version. Eg:

lrwxrwxrwx 1 root root     23 Dec 21  2014 libdbus-glib-1.so.2 -> libdbus-glib-1.so.2.2.2  -rw-r--r-- 1 root root 160832 Mar  1  2013 libdbus-glib-1.so.2.2.2  

This therefore means different version of "libraries" exists in one system - which is not a problem as it is the same file, and it will provide compatibilities when applications have multiple versions dependencies.

Therefore, at the system level, all the libraries are almost interdependent on one another, and just changing the libraries loading priority via manipulating LD_PRELOAD or LD_LIBRARY_PATH will not help - even it can load, runtime it may still crash.

http://lightofdawn.org/wiki/wiki.cgi/-wiki/NewAppsOnOldGlibc

Best alternative is chroot (mentioned by ER briefly): but for this you will need to recreate the entire environment in which is the original binary execute - usually starting from /lib, /usr/lib/, /usr/lib/x86 etc. You can either use "Buildroot", or YoctoProject, or just tar from an existing Distro environment. (like Fedora/Suse etc).


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »