Learnings from moving Kicad to wxPython 4.0

This post is likely only of potential interest to Kicad developers.
Kicad uses wx as its GUI platform and has a python interpreter built in. wxPython is used to bridge between wx and python. Kicad uses SWIG to expose its internal data structures to the Python world.
EDIT: the quote below mis-characterizes the person being quoted. See here for a good clarification of what he meant
I think strong scripting is one the best features many tools can provide but I fear that the kicad development team 1 doesn’t share in my enthusiasm. One of the reasons given why python support may be dropped is the need to migrate to wxPython 4.0, which is a rewrite of the previous 3.0.
https://lists.launchpad.net/kicad-developers/msg31672.html

Another thing, there will be no more wxPython releases. The project has been renamed to Phoenix and AFAIR nobody has tried to use it with KiCad. It may turn out that our SWIG work might be dropped/converted or we will stay stuck with wx3.0 forever.

I encourage everyone that finds the python interface to kicad valuable to post to the kicad developers list.
So I tried running Kicad with the Phoenix version of wxPython. It seems to work, but it’s not quite ready to be used in released kicad. As far as I know, the more recent wx versions don’t do anything that breaks the wxPython 3.0 codebase.

The executive summary:

  • Phoenix, as released, is not ready be embedded into applications. It’s usage thus far is as a plugin in Python. Kicad also uses python, but at the top level, it’s c++ main, not python.
    • The only thing that’s missing is that the file src/wxpy_api.h isn’t copied from the source tree to Phoenix releases. See here.
  • Phoenix uses SIP instead of SWIG to expose c++ to python. Kicad uses SWIG.
    • My experience from this experiment is that SWIG and SIP can coexist, but my testing is limited. The kicad python scripts from my github all worked.
    • Some of the kicad APIs visible in Python involve wx pointers to points, boxes and such 2. Seems to work fine.
  • In my kicad version using Phoenix, the wx stuff seems to work fine. I have some scripting that binds callbacks for mouse and key events. Seems to work.
  • When installing Phoenix, you have to be careful of whether you have gtk2 or gtk3 on your system. Perhaps this was an issue before and I just accidentally installed the right one.

Needed code changes

There are only a handful changes needed to kicad’s code base to make it work with Phoenix. In the code snippets below, I give the original code, commented, followed by my replacement. Note that I didn’t try to do it “correctly”. I just wanted to get it to work.
Here’s a diff file of the changes.
  • CMakeLists.txt wants the wx version number to match the wxpython version. I just hacked it by hard coding the wxpython version to 4.0
[code]
#set( _wxpy_version “${wxWidgets_VERSION_MAJOR}.${wxWidgets_VERSION_MINOR}” )
set( _wxpy_version “4.0” )
[/code]
  • CMakeLists.txt uses wxversions package to select the wx version. That subpackage is gone. Those wanting to have multiple wxpython versions installed should use virtual python environments.
[code]
#set( _py_cmd “import wxversion;print wxversion.checkInstalled(‘${_wxpy_version}’)” )
set( _py_cmd “import wx; import re; print(re.search(‘wxWidgets 3.0’, wx.wxWidgets_version) != None)”)
[/code]
  • pcbnew/swig/python_scripting.cpp requires several changes.
First, it also uses wxversions
[code]
//snprintf( cmd, sizeof(cmd), “import wxversion; wxversion.select(‘%s’)”, WXPYTHON_VERSION );
snprintf( cmd, sizeof(cmd), “import wx”);
[/code]
Next, wxPyCoreAPI_IMPORT() doesn’t exist anymore. I’m guessing that my change above to just import wx does the same thing.
Since Phoenix uses SIP instead of SWIG, the utilities used to convert between c++ and python have changed
[code]
//PyObject* arg = wxPyMake_wxObject( parent, false );
PyObject* arg = wxPyConstructObject(parent, “wxWindow”, false);
// unrelated code …
//bool success = wxPyConvertSwigPtr( result, (void**) &window, “wxWindow” );
bool success = wxPyConvertWrappedPtr(result, (void**) &window, “wxWindow”);
[/code]
  • pcbnew/python/kicad_pyshell/__init__.py needs to initialize wx:App() and hold a pointer to it. I don’t know where this happened before; it’s not really new. I just added this code:
[code]
# there’s surely a better way to do this. I have to create
# and keep a pointer to the app object. If I don’t store this
# (otherwise unused) object, the garbage collector will delete
# it and other wx stuff will fail.
self.theApp = wx.App();
[/code]
  • pcbnew/swig/python_scripting.h just needs to include the new interface file (which will hopefully be included in wxPython (4.0+ AKA Phoenix) releases)
[code]
// #include <wx/wxPython/wxPython.h>
#include <wxpy_api.h>
#include <wx/window.h>
[/code]

Installation details

One thing I found a little tricky was ensuring that the correct copies of everything are being sourced.
gtk2 vs gtk3
I had gtk2 installed on my machine. The installation instructions example for wxPython is for gtk3. There is a gtk2 version, I just didn’t know to look for that difference. For pip base install, I used something like this:
[code]
pip install -U –pre \
-f https://wxpython.org/Phoenix/snapshot-builds/linux/gtk3/ubuntu-16.04 \
wxPython
[/code]
You can also build wxPython yourself. These instructions worked for me.
If you get an error like this
../src/common/object.cpp(251): assert "classTable->Get(m_className) == NULL" failed in Register(): Class "wxCommandEvent" already in RTTI table - have you used IMPLEMENT_DYNAMIC_CLASS() multiple times or linked some object file twice)?
It means you have gtka based wx installed but wxPython wants a gtkb wx.

some prereqs

To get phoenix to install, I needed some package I hadn’t had before. This post was helpful, though I made these changes:
  • libtiff4 updated to libtiff5
  • leave out libjpeg62-dev to avoid this error:
    libjpeg-turbo8-dev : Conflicts: libjpeg62-dev but 1:6b2-2 is to be installed
leaving me with this:
[code]
sudo apt-get install libgl1-mesa-glx libglu1-mesa libgl1-mesa-dev libglu1-mesa-dev libgstreamer0.10-dev libgconf2-dev libsdl1.2-dev zlib1g-dev libtiff5-dev python-gst0.10-dev
[/code]
That’s it. Hope it was helpful.
Learnings from moving Kicad to wxPython 4.0

  1. I’m not a kicad developer. I’m just someone who knows how to program wanting to contribute

  2. this should change to using tuples instead. why should I call foo(wxPoint(x,y) instead of foo(x,y)?