Getting Python to Go Faster

No one likes a slow application. So, it doesn’t surprise me that readers of Professional IronPython and Beginning Programming with Python For Dummies have asked me to provide them with some tips for making their applications faster. I imagine that I’ll eventually start receiving the same request from Python for Data Science for Dummies readers as well. With this in mind, I’ve written an article for New Relic entitled 6 Python Performance Tips, that will help you create significantly faster applications.

Python is a great language because you can use it in so many ways to meet so many different needs. It runs well on most platforms. It wouldn’t surprise me to find that Python eventually replaces a lot of the other languages that are currently in use. The medical and scientific communities have certainly taken a strong notice of Python and now I’m using it to work through Data Science problems. In short, Python really is a cool language as long as you do the right things to make it fast.

Obviously, my article only has six top tips and you should expect to see some additional tips uploaded to my blog from time-to-time. I also want to hear about your tips. Make sure you write me about them at John@JohnMuellerBooks.com. Be sure to tell me which version of Python you’re using and the environment in which you’re using it when you write. Don’t limit your tips to those related to speed either. I really want to hear about your security and reliability tips too.

As with all my books, I provide great support for all of my Python books. I really do want you to have a great learning experience and that means having a great environment in which to learn. Please don’t write me about your personal coding project, but I definitely want to hear about any book-specific problems you have.

 

 

NumPy and SciPy Update

As books age, some of the resources used to create them get abandoned or simply don’t work right. Such is the case with Professional IronPython. There has been an ongoing conversation about NumPy and SciPy support for the product. In fact, you’ll find the first signs of trouble in my NumPy and SciPy Support in IronPython 2.7 post, followed by an update in NumPy and SciPy Support in IronPython 2.7 – An Update. At the time I uploaded those posts, Enthought was still engaged in producing a NumPy and SciPy library for IronPython. Recently, a reader notified me that the support is no longer available from Enthought—a problem that I’ve since verified.

Of course, my posts alerted to you issues with the Enthought library and things have only gotten worse from what I understand. As a result, I can’t even recommend you download and try the Enthought library an longer, unless you’re running an older version of IronPython and just happen to find a compatible version. There is a version on the SciPy site. However, when you review the FAQ on this site, you see this information related to .NET installations:

Does NumPy/SciPy work with IronPython (.NET)?

Some users have reported success in using NumPy with Ironclad on 32-bit Windows. The current status of Ironclad support for SciPy is unknown, but there are several complicating factors (namely the Fortran compiler situation on Windows) that make it less feasible than NumPy.

So, the chances of the SciPy installation working are relatively small. There are some additional sites you could try, but given that I haven’t actually tried them myself, I can’t guarantee success. Please contact the site owners if you have questions about their software. With this caveat in mind, you can try these locations:

If someone has actually tried these extensions and used them successfully, I’d really like to hear from them. It would be nice to have a working solution for the book. In the meantime, there is a message thread at https://mail.python.org/pipermail/ironpython-users/2014-May/017067.html that could provide helpful information about the situation. Anyone with book-specific questions should feel free to contact me at John@JohnMuellerBooks.com.

 

Examining the Calculator in Windows 7 (Part 2)

A while back, over two years ago in fact, I uploaded a post entitled, “Examining the Calculator in Windows 7.” Since that time, a number of people have asked about the other features that the new calculator includes. Yes, there are these rather significant problems that Microsoft has introduced, but there are some good things about the new calculator as well.

The good thing appears on the View menu. When you click this menu, you see options at the bottom of the list that provide access to the special features as shown here.

The View menu includes options for unit conversion, date conversion, and worksheets.
The Windows 7 Calculator View Menu

The Unit Conversion and Date Conversion options are the most useful. However, the worksheets can prove helpful when you need them. Of the new features, I personally use Unit Conversion the most and many people likely will. After all, it’s not often you need to figure out a new mortgage, vehicle lease amount, or the fuel economy of your vehicle (and if you do such work for a living, you’ll have something better than the Windows Calculator to use). To see what this option provides, click Unit Conversion. You see a new interface like the one shown here:

The Unit Conversion display makes it possible to convert from one unit of measure to another.
Calculator Unit Conversion Display

You start using this feature by selecting the type of unit you want to convert. As you can see from this list, the kinds of conversions you can perform are extensive:

Select a conversion type to determine what options are offered in the From and To fields.
The Calculator Supports a Healthy List of Conversion Types

The option you select determines the content of the From and To fields. For example, if you want to convert from kilometers to miles, you select the Length option. After you select the type of unit, type a value in the From field and select the From field unit of measure. Select the To field unit of measure last. Here is what happens when you convert 15 kilometers to miles:

The output shows that converting 15 kilometers to miles equals 9.32056788356001 miles.
Converting Kilometers to Miles

I’ve found use for most of the entries in the types list at one time or another. Every one of them works quite well and you’ll be happy they’re available when you need them. The Data Calculation option can be similarly useful if you work with dates relatively often, as I do. However, I can’t see many people needing to figure out the number of days between two dates on a regular basic. Even so, this feature is probably used more often than any of the worksheets.

The ability to perform conversions of various kinds and to access the worksheets that Windows 7 Calculator provides isn’t enough to change my opinion. The implementation of the Calculator is extremely flawed and I stick by my review in the first posting. However, you do have the right to know there are some positives, which is the point of this post. Let me know your thoughts about Calculator now that you have a better view of it at John@JohnMuellerBooks.com.

 

NumPy and SciPy Support in IronPython 2.7 – An Update

Since I last wrote about NumPy and SciPy support in IronPython (see the NumPy and SciPy Support in IronPython 2.7 post), I’ve received some additional e-mails from readers on the topic. It turns out that there are some twists and turns to this support issue. Some readers have noted that the use of ironpkg-1.0.0.py file from Enthought actually causes problems with the Visual Studio IDE. You may very well see an assertion error. The error shows up when you attempt to type clr.AddReference(. As soon as you type the (, the IDE displays the assertion error. I haven’t personally experienced the error, but you can find discussions of it at http://pytools.codeplex.com/discussions/349615 and http://pytools.codeplex.com/workitem/656 (amongst other locations).

The issue is that the NumPy and SciPy support is strongly compiled to use a specific version of the IronPython libraries. In fact, a few authors have gone on to suggest that you use the version of IronPython supplied by Enthought, which seems like a good idea to me. I’m using IronPython 2.7.3 downloaded directly from the Codeplex site without any problem. For the time being, I’ll continue to use this version for your support questions so that I can provide the best possible information. If you encounter problems using this version, you may have to download the Enthought-specific version of IronPython to ensure NumPy and SciPy support works (or go without these libraries).

Simply installing the Enthought version of IronPython won’t be enough to fix the problem, unfortunately. You must remove the NumPy and SciPy library files completely using the ironpkg –remove command (specifically ironpkg scipy –remove in this case). You must then uninstall the copy
of IronPython that caused the error, which includes physically removing
the folder that contains the IronPython files. After you remove everything, restart Visual Studio to ensure that your IDE is working properly. Then you can install the Enthought version of IronPython and reinstall the NumPy and SciPy libraries. Following this procedure should fix the assertion error if you’re experiencing it.

I’ve been told that there will eventually be a fix for this problem, but when you think about it, the fix can’t be complete using the current programming strategy because you’re relying on low level routines to perform a specific task. Remember that you must start your application using the -X:Frames switch or you’ll most definitely run into problems. Let me know if you encounter any other issues with this particular issue at John@JohnMuellerBooks.com.

 

A Common IronPython References Error

I’ve been encountering a common error when answering reader emails for Professional IronPython. The problem is compounded by the fact that the error messages readers receive vary and there seems to be little consistency in the way Visual Studio reacts to the error. When making references to a .NET assembly in IronPython, you must include the .DLL extension. Otherwise, the IDE is going to give you a very odd error message nearly every time. For example, if you want to reference the System.Drawing assembly, you’d use the AddReference() method like this:

import clr
clr.AddReference('System.Drawing.DLL')

When working with the book, Chapter 7 tells how to interact with the .NET Framework. In fact, you can find the procedure for importing a CLR assembly on page 125. In addition to importing clr, you also need to provide a path for finding the assemblies as shown in the example code.  Chapter 8, page 143, shows a simple example of a Windows Forms application. Most readers find the code in Listing 8-1 really helpful.

Making matters worse, some readers have told me that they have been able to make applications work without including the .DLL part of the file name, but I’m finding including the whole file name works better. These odd errors are a concern for me, so please let me know if you continue to experience problems with IronPython, Python Tools for Visual Studio (PTVS), or the use of NumPy and SciPy at John@JohnMuellerBooks.com. I’ll try to reproduce the error on my system so that I can troubleshoot it with greater ease. If I can’t reproduce the error, I’ll likely need some additional input and testing from you.

 

NumPy and SciPy Support in IronPython 2.7

On page 434 of the English version of Professional IronPython, I talk about some much needed CPython additions to IronPython. Two of these additions are the Numerical Python (NumPy) and Scientific Python (SciPy) packages. You use these two packages in tandem to perform scientific numeric computations. In other words, a lot of people use these packages to perform heavy duty math. At the time of writing, you needed to obtain a copy of the IronClad product (discussed on page 437) to make the packages work. That’s because IronPython has trouble supporting some CPython packages.

Recently, a reader mentioned that he was having trouble obtaining a copy of the required IronClad product for IronPython 2.7. A quick search didn’t point out any potential solutions to the problem. Of course, you can always continue to use IronPython 2.6 for development purposes, but then you lose out on all the great new features in IronPython 2.7 (see IronPython 2.7 and PTVS and IronPython 2.7.1 Update). To make things a little more complicated, there is a new IronPython 2.7.3 release to consider as part of this question.

I did quite a bit of research about this issue. IronClad is definitely aware of the problem because people have been writing them about it. A second solution that entails using a product from Enthought doesn’t work because the required download is missing. I currently have e-mails in to Enthought and hope that I receive a response that I can pass along. In short, the bad news is that you can’t use NumPy and SciPy with IronPython 2.7 for now, but there should be a solution available. More than a few people have said they were able to get the Enthought solution to work—all we need is the Enthought solution to become available.

Even though the binaries are missing, I did manage to find source files at https://github.com/ilanschnell/ironpkg. The only problem is that I know nothing at all about these source files or how to build them into something you can use with .NET. The README.txt file does make it clear that these are the correct files. (Interestingly enough, I found the required egg files at http://www.enthought.com/repo/.iron/eggs/, so most of the support seems to be there.)

I have found one bit of useful information about employing this solution, you need to use it with the -X:Frames command line switch. The Enthought documentation doesn’t make this very clear and some users have encountered problems making the solution work. If you find out where we can locate the missing ironpkg-1.0.0.py file, please let me know at John@JohnMuellerBooks.com.


Breaking news update! I’ve just tried downloading the ironpkg-1.0.0.py file from Enthought and it now works as anticipated. If you need NymPy and SciPy support in IronPython 2.7, please use the Enthought solution. Make sure you follow the directions precisely, including the need to use the -X:Frames command line switch. I found that I did need to import SciPy once inside the interactive environment, rather than at the command line as suggested on the Enthought site, but that could be a problem with my system. Contact me at John@JohnMuellerBooks.com with the result of any programming you do, but as far as I can tell, this solution now works as anticipated.

 

Thinking Through Indentation and IDE Automation

I’ve been asked a number of times about code indentation in my books by publishers, editors, and readers alike, so I thought it might be a good idea to talk about the matter in a blog post. It’s important to indent your code to help make the code more readable—to help make the flow of your code easier to see. Each indentation represents another code level. The application is at one level, classes at another, methods within the class at another, loops within a method at still another, and so on. By viewing the various levels of code within an application, you see an outline of application functionality and can better understand how each application element works. Because indentation is so important to the understanding of code, I spend considerable time working through indentation in my books so that you can better understand the examples I present.

Indentation is there for the developer to use as an aid to understanding. A few early computer languages, such as COBOL, were positional and depended on coding elements appearing in certain positions on each line. Most languages today don’t require that you use any sort of indentation. In fact, more than a few languages would let you write the entire application on a single line without any indentation at all (as long as you supplied a space between statements and expressions as needed). The compiler doesn’t care whether you indent using spaces or tabs, whether there are three or four spaces per indent, or whether you provide an indent for continued lines. The fact is that all of these characteristics are controlled by the developer to meet the developer’s needs.

Modern IDEs make it easy to indent your code as needed by performing the task automatically in some cases. For example, given an example of a Java method, you could begin by typing:

public void MyProc()
{

The moment you press Enter after the curly bracket (}), most IDEs will add an indented line and a closing curly bracket. Your cursor will end up on the indented line, ready for you to type a statement. You see something like this in your IDE:

public void MyProc()
{
 
}

The developer hasn’t done anything so far to provide indentations for the code, yet the code is already indented. Let’s say you create a variable like this:

boolean MyVar = true;

You then type a statement like this:

if (MyVar == true)

It’s at this point where some IDEs continue to provide an automatic indent and others don’t. For example, if I press Enter at the end of this line in Eclipse, the IDE automatically indents the next line for me. However, when I’m working with CodeBlocks, pressing Enter leaves the code at the same indention level as before. Neither approach is wrong or right—simply different. If I were to add a curly bracket on the next line to hold multiple lines of code within a block, CodeBlocks will already have the cursor in the right position and Eclipse will have to outdent the curly bracket to position it correctly.

Let’s say that this if statement has just one line of code following it, so I don’t use a curly bracket. When working with Eclipse, the cursor is already at the correct position and I simply type the next statement. However, when working with CodeBlocks, I must now press Tab in order to indent the line of code to show that this line (and only this line) is associated with the if statement. In this case, the example simply outputs a statement telling the value of MyVar like this.

System.out.println("MyVar == true");

Pressing Enter automatically outdents the line when using Eclipse because the next statement is automatically at the same level as the if statement. However, when working with CodeBlocks, I must press the Backspace to outdent the line manually. The resulting method could end up looking like this:

void MyProc()
{
    boolean MyVar = true;
    if (MyVar == true)
        System.out.println("MyVar == true");
     
}

Of course, a good developer will add some comments to this code as a reminder of what task the code performs and why the developer chose this particular technique to perform the task. The point of this post is that indentation is an essential part of working with most languages in order to make the resulting application easier to understand.

A final thought on IDEs is that most of them make it possible to configure the editor to indent or not indent to meet the requirements of your organization or personal tasks. IDEs commonly allow the use of tabs or spaces for indents (spaces are better when you want to write documentation). You can also choose the right amount of indentation (three spaces is optimal for books where space is limited). Let me know your thoughts on indentation and how you use it at John@JohnMuellerBooks.com.

 

Configuring Visual Studio for IronPython Use

Last week I discussed a small bit of information I left out of my book in the procedure that begins on page 26 (see Professional IronPython Chapter 2 Step-by-Step Procedure Update for details). It turns out that the small bit of information isn’t the only thing you need to worry about with that procedure. A reader wrote in to mention that the basic settings you choose can also make a difference. If you have Visual Studio configured for Web Development, the procedure won’t work. That’s because of the way Visual Studio treats Web development. The problem is that you won’t have a solution (.SLN file) to save. I encountered this problem once before, but forgot about the issue until the reader wrote to me. Use the following procedure to check your settings:

  1. Choose Tools→Import and Export Settings. You’ll see the Import and Export Settings Wizard dialog box shown here.
    VisualStudioSetup01
  2. Choose Reset All Settings and click Next. The wizard will ask you to save your current settings as shown here. It’s always a good idea to save your settings before even looking at the configuration.
    VisualStudioSetup02
  3. Click Next. You’ll see the Choose a Default Collection of Settings page shown here. This is where the problem lies. I normally choose General Development Settings because I work in a lot of different languages. If your dialog box shows that Web Development is selected, then the procedure won’t work.
    VisualStudioSetup03
  4. Choose a setting other than Web Development (I highly recommend the General Development Settings as being the most versatile) and click Finish. Visual Studio will reconfigure itself for whatever settings you chose. This change will make it possible to save the project solution file for the examples in Professional IronPython.

 

If anyone encounters any other problems with this procedure, please be sure to let me know at John@JohnMuellerBooks.com. I’m always happy to hear from readers about issues regarding my books and want to make things as easy as possible for you.

Professional IronPython Chapter 2 Step-by-Step Procedure Update

Every once in a while, a small error gets past me that really is almost microscopic, yet ends up causing readers woe. Errors of this sort (or any sort at all) are the reason I wrote the Errors in Writing post so long ago. In this case, there is a small mistake in Professional IronPython that might not seem like such a big deal at first, but it has caused problems for readers, so I’m providing an update here.

In Step 7 on page 26 I ask you to choose File→Save All, after which, you’re supposed to see the dialog box shown in Figure 2-4. It turns out that the step doesn’t quite work and for an important reason. Back in step 4 I asked you to right click the IPY entry in Solution Explorer. Of course, that step highlights IPY, selecting it. As a consequence, step 7 can’t work as written because you can’t save IPY.

To make Step 7 work as printed in the book, first highlight the Solution ‘ipy’ (1 project) entry in Solution Explorer (see Figure 2-2). Selecting this entry means that you’re asking Visual Studio to save the solution, not IPY. Now when you choose File→Save All, you’ll see the Save File As dialog box shown in Figure 2-4 as expected.

Please let me know if you encounter any other errors of this sort and I’ll be more than happy to discuss them in my blog. For this, and any other book-related concerns, contact me at John@JohnMuellerBooks.com.

 

Visual Studio 11 and Windows 8

The trade press is beginning to talk about Visual Studio 11 quite a bit more often now, even though the release of the initial beta won’t even happen today (coincident with the consumer release of Windows 8). The word on the streets is that you won’t see anything too exciting or unexpected in Visual Studio 11. Yes, Microsoft is making the usual laundry list of changes, but there isn’t anything that you wouldn’t expect them to add. Here’s a quick overview of the changes that I’ve been able to glean from the various articles I’ve read:

 

  • HTML5 and JavaScript support
  • Touch application support
  • Agile application development
  • Cloud application development
  • .NET Framework 4.5 support
  • Metro interface designers
  • Devops integration (a tool designed to make it easier for administrators and developers to communicate)
  • IntelliTrace updates that make it possible to trace activity in operational applications
  • Asynchronous programming updates
  • Parallel program development
  • Improved SharePoint support


In other words, Visual Studio 11 is targeting Windows 8 and Windows Azure development. Along with these changes, there are rumors that Microsoft will simplify the IDE to make it easier to use. I just hope that they don’t bury functionality in places where you can’t find it. The Ribbon interface that Microsoft is so fond of adding to other applications does just that—making them a nightmare for power users to use, while simultaneously reducing the learning curve for the novice (the bad and the good that comes with any change).

There isn’t any mention of one of my pet peeves with Visual Studio 10, it’s voracious appetite for system resources and an unflinching desire to freeze the system. The current version of Visual Studio tends to require me to reboot my system at least once during any programming session.

Also missing from the laundry list is any mention of Silverlight, which makes me wonder whether Microsoft is putting this technology out to pasture. In addition, there was nary a mention anywhere of updated Windows Phone support. You’d think that Microsoft would be providing some updated support for Windows Phone development considering that many developers will need to support the mobile device. More and more users are augmenting their desktop systems (or replacing them entirely) with mobile devices, so Microsoft had better get on the ball.

Everything I’m reading focuses on Microsoft’s two main languages C# and Visual Basic. For example, I haven’t seen anything about F#, which appears in the current product. Of course, IronPython was orphaned quite some time ago, but the IronPython community remains alive and well (see my IronPython 2.7.1 Update post for details). This lack of information makes me wonder about the future of Microsoft’s other languages (including C++).

Of course, the updates in Visual Studio 11 and the .NET Framework 4.5 will affect applications of all sorts. During the upcoming months I’ll try to test all of the application examples in my books to determine whether there are any issues you need to know about. In the meantime, please continue to use the supported versions of Visual Studio when working through the examples in my books to ensure you have the best learning experience possible. Contact me with any concerns or observations you have about Visual Studio 11 at John@JohnMuellerBooks.com.