人生不设限的意思:How to build matplotlib on Cygwin, undaunted ...

How to build matplotlib on Cygwin, undaunted by build errors or the impending shadow of death.

matplotlib takes the goodness of the Matlab plotting API (and more!) and puts it in Python. It’s an astonishingly identical API. And now you want to compile it on Cygwin. You’ve scoured the Usenets. You’ve scoured your pots and pans. And it’s impossible. Or is it?

(This has been tested on matplotlib-0.98.5.2, February 21, 2009, on an absolutely up-to-date Cygwin. Expect to improvise more and more for later and later versions. Feel free to add any additional tips or questions to the comments section. This is the magic of webbogs.)

Well, it probably isn’t, since the matplotlib people can do it or otherwise it wouldn’t be in their setup.py. But things break, and manly administrators like us need to fix them.

Grabbing the source.

First, run easy_install, which will automatically find the right .tar.gz file from SourceForge. At that point, abort with Ctrl-C and copy and paste the URL. We won’t be using easyinstall because we’ll be fidgeting around in the building files.

Now choose a directory. wget and untar the file into this directory.

Normally, we’d run python setup.py install and go grab a cheesecake. Not so fast. You’ll get errors aplenty.

As proof, run python setup.py install. It will raise an IndexError in setupext.py in trying to read from your /usr/lib/tkConfig.sh in the function parse_tcl_config. It seems to expect a different format than what’s actually present. Fortunately, the guess_tcl_config function is Good Enough. All we have to do is force the function to enter the guess_tcl_config code path:

To swallow this exception, add “IndexError” to the comma-delimited list beneath that try-block. In other words, turn except (ConfigParser.NoSectionError, ConfigParser.NoOptionError): into except (ConfigParser.NoSectionError, ConfigParser.NoOptionError, IndexError):. Exception swallowed!

Now run python setup.py install again like you did before. We’ll need to make sure we have all the dependencies, so let’s see what the library sniffer picks up. Install anything, via Cygwin, that it can’t find.

Two notable dependencies it won’t pick up: libX11-devel and pkg-config. Make sure you have these before running the library sniffer because you’ll need to save up your insanity for this build. Keep running setup.py and keep installing until you resolve all the dependencies.

An initial failure in building C extensions.

Here’s the crux of the problem with setup.py: You’ll get weird fork resource errors when setup.py tries to compile the C/C++ extensions. Happily enough, Pete Harris noticed that this doesn’t happen if you manually run gcc/g++. A mystery, one that can’t be solved with Cygwin’s rebase, and one that we don’t have time for.

A simple solution is to simply mimic setup.py’s calls to gcc from our own prompt. setup.py (distutils, to be exact) is smart enough to not recompile when it’s not necessary, which makes this fast and easy to implement. The only tricky part is getting setup.py to spit out all its gcc forks, not just the one it dies on.

To review: How do we get setup.py to tell us all the gcc commands it’s going to run, without actually running them?

Squeezing a build script out of distutils.

For that, we turn to python setup.py --dry-run build > a.txt or, shorter, python setup.py -n build > a.txt. We’re piping this to a.txt because there’s a lot of junk to remove. It should look like this. Once setup.py finishes, open a.txt in your favorite text editor, which we’ll call “Emacs”, and delete everything around the gcc/g++ chunks.

Within this chunk, there are messages to the user such as “building such-and-such” or “creating such-and-such”. Delete those. And slap a sh shebang at the top to get a file like this, which you can save to matplotlib-clean-build.sh if you so choose. Now you have a build script.

At this point, you’re eager to run the script and see all that glorious gcc output, a crimson-violet rainbow of warnings and C++ templates and library paths. But we have more to do.

Invalid file paths.

For some reason, there are paths to old files in this output, files that have since then been renamed. In particular: src/backend_agg.cpp, src/image.cpp, and src/path.cpp. These do not exist. Find the occurrence of each and add an underscore the file name. For example, it should be src/_path.cpp, not src/path.cpp. Warning: stick to .cpp files only. The .o files, which are intermediate output filenames that nobody cares about, are fine.

On the very last line, when g++ tries to build _tkagg.dll, change -ltk8.4 to -ltk84. Similarly, change -ltcl8.4 to -ltcl84. Cygwin maintainers and matplotlib developers have different ideas about how these two libraries should be named in your /usr/lib.

To recap: distutils thinks there should be files where there are none because they were renamed. We are here to fix it.

End game.

Now run the build script with ./my-build.sh and then run python setup.py install, which will copy the Python files and your built extension—that you lovingly built by hand! it’s your children!—to all the right places.

Most likely, though, you’ll encounter other errors that I never ran into. Google and smart guessing are your friends.

The worst enemies of build errors are: reading, Googling, logical reasoning, acking or grepping, a good text editor, lathering, rinsing, and repeating.

Great. Thanks a lot. It worked fine for me on matplotlib-1.0.0 I had to create a few directories manually though at the compilation step.

Thanks, Dan! Glad to hear that it still semi-works on recent matplotlib versions.

Thanks very much! I have successfully installed matplotlib-1.0.1 on Cygwin without any further changes.

Thanks a lot. It was much easier to install and use cygwin matplotlib in cygwin xterm.

Thought I’d post my experiences for those who come after.

These instructions still work with matplotlib-1.0.1 on Cygwin with python2.6, but for whatever reason I was encountering a fork error at the point where setup.py is performing the optional USETEX dependency checks. This prevented the setup script from reaching the —dry-run option. I managed to work around this by:

1. Force (uncomment) tkagg = False etc. in the setup.cfg file
2. Run python setup.py —dry-run build > a1.txt
3. Run python setup.py build so that the majority of the script can compile, create directories etc.
4. Comment out the tkagg = False etc. in the setup.cfg file
5. Comment out check_for_dvipng(), check_for_ghostscript(), check_for_latex(), check_for_pdftops() in setup.py
6. Run python setup.py —dry-run build > a2.txt
7. diff a1.txt a2.txt to get a list of the additional gcc/g++ lines that still need to be run and run these either in a script or just copy pasting to the command line

To install, I had to jump through similar hoops:

1. Undo the changes of step 4 and redo step 1.
2. Run python setup.py install
3. Redo the changes for step 4
4. Run python setup.py install

I hope this helps someone, and if not I have a nice set of notes if I ever need to repeat this traumatic exercise. :)

I was able to use most of the same steps to compile matplotlib-1.1.0:

1) Unpack your source and change into source directory:

$ tar xvzf matplotlib-1.1.0.tar.gz$ cd matplotlib-1.1.0

2) parse_tcl_config in setupext.py will fail when running `python setup.py build’, but making it return None will make other mechanisms take over, so insert as the first line with your editor or do:

$ sed "/def parse_tcl_config/a\    return None" -i setupext.py

3) calls to gcc/g++ from `python setup.py build’ will fail, but extreacting into a script and running that will work:

$ python setup.py -n build | grep -E '(g\+\+|gcc)' > compile.sh

4) move some source files into place, run compile.sh, run the build script, and repeat this until the build script doesn’t call gcc anymore. I believe you have to call it 11 times, to build all the dll’s. Maybe it should be put into a loop to call it 11 times automatically…

$ for file in src/_*; do cp ${file} ${file/\/_/\/}; done ; sh compile.sh ; python setup.py build

5) switch to an administrator-started cygwin-shell and finish the installation:

# python setup.py install

Step 4) is an approximation of what I did. Basically compile.sh seems to contains 11 sections of compilation commands, where each section ends with a line ending in “.dll”. If you split up compile.sh into those 11 section, and substitute them one by one into step 4 instead of the full compile.sh, it should work and this is closer to how I managed it. Maybe the next person can put together a magic incantation for that ;P