Python

I like Python because it's fun to program in. The syntax is intuitive and easy to read; a lot of people describe it as "clean". There are enough libraries available to get you going quickly with all sorts of tasks. It works well for both simple scripts and complex programs, and advanced programming techniques are possible, such as meta-programming. I've found it to be very stable and reliable; I have yet to see the interpreter crash with an internal error. There are concerns with performance and memory consumption, but I've always found it to work fine, even for bulk data tasks.

If you want to learn Python, start with the tutorial on python.org. After that, you really need a project to work on, to focus your direction. As you try to code your project, you'll work through all sorts of wacky solutions to problems, gradually figuring out good ways to do things. Also, reading other people's code is a great way to pick up new tricks.

The standard library takes care of many things. There are language services such as threads, dates and regular expressions. Moreover, there are modules to help with common tasks, such as talking HTTP and SSL, working with CSV files and formatting text. And beyond this there are third-party libraries for a great range of tasks. Here are some of my favorites.

wxPythonwxPython is a windowing toolkit. It lets you create native look-and-feel Windows, Mac and Linux GUI applications. It supports a wide range of features, and offers the programmer a lot of control. A rudimentary RAD system is available with XRCed. The syntax is reasonably in line with other Python code, although another library called Dabo is available, which sits on top of wxPython and aims to make the syntax more "Pythonic".
PyWin32This collection of libraries gives you access to Windows-specific functions. One of the most powerful components is win32com that gives you full access to COM. You can use this to do all sorts of tasks, including access data in Microsoft Office documents.
SendKeysSendKeys lets you send fake key strokes to Windows programs. I've used it to automate calling programs that don't have a command line interface, e.g. to generate PGP self-decrypting archives.
py2exepy2exe packages up a Python program as a stand-alone .exe that can run on a computer that doesn't have Python installed. It's worked very reliably for me, and a program comes out at around 3 Megabytes. Also, InnoSetup is a free solution to create distributable Windows installers.
ToscaWidgetsThis is a library of re-usable web components. It allows client-side and server-side functionality to be bundled into widgets. This makes it much easier to build dynamic, interactive websites.
SQLiteSQLite lets you have a mini SQL database embedded in your program. I find this greatly preferable to other peristance techniques, such as pickle and shelve. It doesn't require any database program to be running on the computer, it just does the work inside your program and stores data in a file. The support for SQL features is very good, and it supports basic multi-user locking. sqlite itself isn't Python-specific; PySQLite is the Python library, and it is built-in with Python 2.5. Also, SQLCipher is a variant that lets you encrypt the database.
SQLAlchemySQLAlchemy is an Object-Relational Mapper (ORM) - a convenient way to access SQL databases from Python. It does have something of a steep learning curve, but once mastered it's much better than writing lots of SQL queries. There is support for the most common databases; an appropriate DBAPI module is required for each.
pyODBCThis module lets you access ODBC through the Python DBAPI. It is used by SQLAlchemy for Microsoft SQL Server support. I also use it directly to use Access databases, which SQLAlchemy doesn't support.
GenshiGenshi is an XML templating language. It supports variable substitution, loops, conditional sections and reusable blocks, as well as more advanced features. I find this by far the best way for programs to generate HTML output; I'll never go back to embeddeding HTML fragments in my code.
JPEGThis is a library for reading and writing the metadata stored in JPEG files, e.g. the date the picture was taken, and the camera orientation. It's been very useful for managing my travel photos.
Python Imaging LibraryThe Python Imaging Library (PIL) is a powerful set of graphics functions. It supports conversion between many bitmap formats, manipulations such as resizing and filters and also generating graphics. It's notable that the manipulations work at high quality - it uses good algorithms.
PloticusPloticus is a graphing toolkit. It lets you code scripts that can generate graphs from dynamic data. The scripting language takes some getting used to, but it is very flexible and will do a lot of work for you. Ploticus isn't python-specific; in fact the only Python interface is running ploticus using os.system.
xmltrampxmltramp offers a simple interface to parse and create XML documents. The interface works well with Python; the code looks nice. It doesn't have all the power of traditional XML interfaces such as DOM, but it works for 99% of my needs. There is a similar library called ElementTree that is built-in with Python 2.5.
4Suite4Suite is a fully functional XML toolkit for Python. It allows tight integration with Python programs. It is possible for Python code to call the XSLT processor, and for XSLT functions to be defined that call Python functions. On the downside, the interface is significantly more compilcated than xmltramp.
m2cryptoPython has basic SSL support built-in. However, it doesn't do certificate validation. In other words, when you connect to a server, the server could be an imposter, and despite the encryption you wouldn't know. m2crypto is the easiest way I found to do SSL with certificate validation.
Python Cryptography ToolkitThis gives you low-level access to all sorts of cryptographic functions. In general I recommend against doing this - it's too easy to make mistakes. Most programs should use a high-level library like m2crypto instead. However, this has been great for validating my JavaScript implementations of hash functions.
scapyscapy is a library for decoding and crafting network packets. It gives you complete control over packets, while making common manipulations simple. I find it invaluable as a "tool of last resort" when doing low-level network investigation. I tend to use scapy interactively, not from scripts.
pcapypcapy lets you capture network packets, just like tcpdump. It works great for writing tools like network scanners in Python.
impacketImpacket is another packet library, in a similar vein to scapy. The syntax is a bit more explicit that scapy and there's less "magic", so I prefer Impacket for packet crafting in scripts.

I hope you found this page useful. If you know of any other cool libraries that deserve a place here, please let me know.

wxPython Tips

  • Create the dialogs as XML resources, using XRCed.
  • If you have written your own customized controls, have them as "Unknown" in the XML, and attach them later using AttatchUnknownControl.
  • If you do any blocking calls, its best to use threads, as otherwise you'll block things repaint messages and the app will appear to have crashed.
  • Use the py2exe -w option to make a Windows .exe that doesn't create a DOS box.
  • Create an exception handler that logs to a file. This will greatly aid debugging if problems are reported.
© 1998 - 2012 Paul Johnston, distributed under the BSD License   Updated:12 Jun 2010