IPython Magic Functions

This is an update of a post that originally appeared on April 25, 2016.

All of my current Python language books (and those I collaborated on with Luca Massaron): Machine Learning Security Principles, Algorithms for Dummies, 2nd Edition, Beginning Programming with Python For Dummies, 3rd Edition, Python for Data Science for Dummies, and Machine Learning for Dummies, 2nd Edition allow use of Jupyter Notebook (through Anaconda) or Google Colab to interact with the example code. Both of these IDEs extend the development environment in a number of ways, one of which is the use of magic functions. You see the magic functions in the code of these books as calls that begin with either one or two percent signs (% or %%). The most common of these magic functions is %matplotlib, which controls how IPython Notebook or Jupyter Notebook display plot output from the code.

You can find a listing of the most common magic functions in the Python for Data Science for Dummies Cheat Sheet. None of my books use any other magic functions, so this is also a complete list of magic functions that you can expect to find in our books. However, you might want to know more. Fortunately, the site at https://damontallen.github.io/IPython-quick-ref-sheets/ provides you with a complete listing of the magic commands (and a wealth of other information about Jupyter Notebook).

There are differences in magic function support between Jupyter Notebook and Google Colab, some of which are outlined in our books as needed. None of these differences will significantly affect your learning experience. However, it pays to know that Jupyter Notebook and Google Colab are only mostly the same, not precisely the same, and you’ll encounter differences. The screenshots in my books reflect the Jupyter Notebook version supported by that book, so what you might see on your screen when using magic functions in Google Colab may differ from the book.

Of course, you might choose to use another IDE—one that isn’t quite so magical as Jupyter Notebook or Google Colab. In this case, you need to remove those magic commands. Removing the commands generally won’t affect functionality of the code. The example will still work as explained in the book. However, the way that the IDE presents output could change. For example, instead of being inline, plots could appear in a separate window. Even though using a separate window is less convenient, either method works just fine. If you ever do encounter a magic function-related problem, please be sure to let me know at [email protected].

Using Online IDEs

Many readers today want to code anywhere, using any device, and at any time. Obviously, hauling around a desktop system or even a high end laptop is not on the list of things they want to do. I thought it unique when a reader wrote me to say that coding on a smartphone should be included in my books. I have since had several more readers make the same request. So, I’m no longer surprised to hear about the various methods used to produce source code.

Personally, I think squinting while I code would be uncomfortable, but I definitely don’t want to hinder anyone’s learning process, so many (not all) of my introductory books now include a chapter on coding with mobile or online IDEs. For C++ All-In-One for Dummies, 4th Edition the mobile device tool of choice is CppDroid (a mobile IDE for C++). Python books, such as Beginning Programming with Python For Dummies, 3rd EditionPython for Data Science for Dummies, and Machine Learning for Dummies, 2nd Edition, rely on Google Colab (an online IDE). Just out of curiosity, I tried Google Colab using my smart TV and it did work—not that I plan to code using my TV anytime soon. Even though my latest book, Machine Learning Security Principles, doesn’t include any sort of IDE setup chapter, I did test the source code using Google Colab for that book as well.

Any time you change IDEs, you need to test your code to actually ensure that it will work with that IDE. Consequently, during the writing process I carefully tested every example on both my desktop system and the alternative IDE that the book supports. This dual testing process helps ensure you have a great learning experience. Unfortunately, I don’t have the time or resources to test every possible IDE out there. To obtain a good learning experience from my books, you need to choose one of the IDEs that I mention. In addition, it’s essential that you choose the correct version of the IDE, the one that matches the book, because vendors tend to introduce breaking changes into IDEs and compilers during the upgrade process.

When you contact me about an IDE issue, the first thing I need to know is which book (and edition of that book) you have, followed by the operating system and IDE version you’re using. Otherwise, it’s very tough for me to try to help you and I do want to help you within reason. No, I’m not going to try to support the alternative IDE that you like that was created for you by your long lost relative. I’ll only support the IDEs specifically mentioned in the book. That said, I do want to hear your input about IDEs at [email protected]. If you’re encountering a problem with the IDE that’s used in the book, I definitely need to know about it and I’ll post some helpful information about it on this blog.

Source Code Placement

This is an update of a post that originally appeared on October 12, 2015.

I always recommend that you download the source code for my books. The Verifying Your Hand Typed Code post discusses some of the issues that readers encounter when typing code by hand. However, I also understand that many people learn best when they type the code by hand and that’s the point of getting my books—to learn something really interesting (see my principles for creating book source code in the Handling Source Code in Books post). Even if you do need to type the source code in order to learn, having the downloadable source code handy will help you locate errors in your code with greater ease. I won’t usually have time to debug your hand typed code for you.

Depending on your platform, you might encounter a situation the IDE chooses an unfortunate place to put the source code you want to save. For example, a Windows System might choose the Program Files folder, which contains a space and doesn’t allow saving of files unless you specifically override the default rights. Fortunately, modern IDEs do manage to avoid many of these problems, but you still need to be aware that they could exist, especially when using an older IDE.

My recommendation for fixing these, and other source code placement problems, is to create a folder that you can access and have full rights to work with to store your source code. My books usually make a recommendation for the source code file path, but you can use any path you want. The point is to create a path that’s:

  • Easy to access
  • Allows full rights
  • Lacks spaces in any of the pathname elements
  • On a local drive, rather than a cloud drive in many cases

As long as you follow these rules, you likely won’t experience problems with your choice of source code location. If you do experience source code placement problems when working with my books, please be sure to let me know at [email protected].

Spaces in Paths

This is an update of the previous post originally created on April 20, 2016.

A number of readers have recently written me about an error they see when attempting to compile or execute an application or script in books such as, C++ All-In-One for Dummies, 4th EditionBeginning Programming with Python For Dummies, 2nd Edition, Python for Data Science for Dummies, and Machine Learning for Dummies, 2nd Edition. Development environments often handle spaces differently because they’re designed to perform tasks such as compiling applications and running scripts. When you see an error message that tells you that a file or path isn’t found, you need to start looking at the path and determine whether it contains any spaces. The best option is to create a directory to hold your source code and to place that directory off the root directory of your drive if at all possible. Keeping the path small and simple is your best way to avoid potential problems compiling code or running scripts.

The problem for many readers is that the error message is buried inside a whole bunch of nonsensical looking text. The output from your compiler or interpreter can contain all sorts of useful debugging information, such as a complete listing of calls that the compiler, interpreter, or application made. However, unless you know how to read this information, which is often arcane at best, it looks like gobbledygook. Simply keep scanning through the output until you see something that humans can read and understand. More often than not, you see an error message that helps you understand what went wrong, such as not being able to find a file or path. Please let me know if you ever have problems making the code examples in my books work, but also be sure to save yourself some time and effort by reading those error messages. Let me know if you have any thoughts or concerns about spaces in directory paths at [email protected].

Code::Blocks on the Mac

A lot of Mac users have written to complain about the stability of Code::Blocks 8.02 on the Mac. This is the version used for the 2nd Edition of C++ All-in-One for Dummies. My first recommendation is that you obtain a copy of C++ All-In-One for Dummies, 3rd Edition if at all possible. This edition of the book contains additional installation details, updated examples, and all sorts of extras that will make your C++ learning experience so much better. Of course, not everyone will want to make the upgrade, but I stick by previous posts saying that some examples won’t work as well as they might if you use a different version of Code::Blocks than specified in the books. However, I also feel your pain. I personally didn’t experience stability problems with the 8.02 release and I’m sure others didn’t either, but enough people have complained that I feel obliged to discuss the issue in a post.

The Code::Blocks 13.12 version used for the 3rd Edition book is considerably more stable than the 8.02 version used for the 2nd edition book. If you really must continue using the 2nd edition book with your Mac, I suggest that you update to Code::Blocks 13.12 if you find that the 8.02 version causes you problems. If you go this route, please be sure to read the Using Code::Blocks 13.12 with C++ All-in-One for Dummies post. It provides you with information you absolutely must have in order to use the updated version successfully.

I always want to hear your book-specific input at [email protected]. Your input helps me create better books and it also allows me to provide posts like this one that help readers work around potential issues. Thank you for your continued support of my books!

 

Choosing a C++ Editor

A lot of people have asked why I chose Code::Blocks as the editor for C++ All-In-One for Dummies, 3rd Edition. There are a number of reason that I chose this particular editor including:

  • Ease of use
  • Free download
  • Runs on all the platforms the book supports

However, the fact that Code::Blocks works well for the book and for most beginning projects doesn’t mean it’s the best solution for your particular needs. There are many different C++ Integrated Development Environments (IDEs) out there and I’m constantly trying new products. When the new version of Visual Studio comes out, you can be sure I’ll obtain a copy and check it out because some of my readers use Visual Studio. The problem with Visual Studio is that it tends to use Microsoft additions to the C++ language and it also doesn’t run on all the platforms that the book must support. However, if you’re working with Microsoft systems and need to create a relatively large project, Visual Studio might be a good choice for you—only you can make that determination.

The Eclipse IDE is another good selection. In fact, I used Eclipse when writing Java eLearning Kit for Dummies because it runs on every platform I needed to test. When I wrote this book, I chose to focus on the language though and not really mention the IDE at all. I didn’t want readers to have preconceived ideas of how Java should look inside an editor. I had thought about using Eclipse for C++ All-in-One for Dummies, 3rd Edition, but after reviewing the IDE and comparing it to Code::Blocks, I felt Code::Blocks was a lot simpler. Even so, if you need great multiplatform support for your C++ projects, Eclipse is a great choice.

I recently tried another IDE, Intel Composer. This is most definitely not a IDE for the faint of heart or the light of pocketbook (the asking price is $1,199.00). Of course, many of you are going to question my sanity for even downloading such a product when there are so many less expensive solutions out there. The main reason to obtain a product like this one is that it provides phenomenal parallelism support for multiprocessor applications. In other words, you use this sort of IDE for high end projects. You can read all the other features this product offers on the vendor site. One of the other items that grabbed my attention is that it provides both multiplatform (Windows, Mac, Linux, and Android) and multiformat (Phone, Tablet, PC, Ultrabook, and Server) support. Whether this particular IDE makes sense for your needs depends on the kind of applications you create.

Although the information in Wikipedia is often suspect, you can find a comparison of various IDEs at http://en.wikipedia.org/wiki/Comparison_of_integrated_development_environments. The best way to choose a C++ IDE is to look for a product that meets your needs and then try it out on a subset of the problem you’re trying to solve. Researching the IDE you use is essential because a mistake can cost you a lot of time later. Not every IDE support every C++ feature, every platform, or every team need. Tell me why you think I should move to something other than Code::Blocks for the next edition of the book at [email protected].

 

Working with Eclipse for Windows Developers (Part 2)

In Working with Eclipse for Windows Developers (Part 1), you discovered how to get Eclipse installed and setup for use on a Windows system. It’s a simple process, but it’s also much different than a standard installation, so it’s quite understandable that many people will become confused by the installation requirements. In today’s post, you see how to open the code for a book. Again, it’s a simple process once you know the secret handshake, but Eclipse doesn’t quite do things in the way Windows users expect in all cases.

When the previous part of this series ended, you should have created as Start menu entry and possibly a Desktop or Taskbar shortcut for Eclipse. Use any of these links to open a copy of Eclipse now. You’ll see a dialog box like the one shown here.

Eclipse0201

Eclipse uses the workspace folder to access the project you’re working on. All of the files for a particular book will be stored in a central folder. For example, when you work with the Java eLearning Kit for Dummies, the examples will appear in a folder named Java eLearning Kit for Dummies.

Click Browse and you’ll see a Select Workspace Directory dialog box where you can select the folder you want to use. This folder must appear on your hard drive, not on a DVD or other media that you can’t write data to. In addition, you must have the proper rights to write data to the directory because Eclipse will do just that as you work with it.

Eclipse0202

Select the folder you want to use and click OK. You’ll now see the folder location in the Workspace field of the Workspace Launcher dialog box. Click OK again. At this point, you’ll likely see a dialog box with a bunch of icons in it for doing things like working through the Eclipse tutorial. Notice that one of the options will take you to the Eclipse Workbench as shown here.

Eclipse0203

Click the Workbench icon. You’ll see a blank Workbench, like the one shown here, which doesn’t seem particularly useful.

Eclipse0204

Right click in the Package Explorer and choose New | Java Project. You’ll see a New Java Project dialog box. Type Chapter 01 (or whatever the chapter folder name is for the particular book) in the Project Name field as shown here.

Eclipse0205

Click Finish. You’ll see the source code in Chapter 01 added to your workspace so that you can now interact with the examples in the book. Here is what your Project Explorer could look like now (again, it may be different depending on the book).

Eclipse0206

To open one of the source files, just double click its entry in  Package Explorer. You’ll see the project opened in the editor window where you can examine its code.

You have several options for running an example. The fastest is to right click the example entry in Package Explorer and choose Run As | Java Application from the context menu. You may see a dialog box telling you about certain conditions and asking if you want to proceed. Click Proceed. If you have the file open in the editor, you can choose Run | Run to run the application. In either case, you’ll see the Console window become active next as shown here for the SayHello.java example from Chapter 01 of my book.

Eclipse0207

The Console window displays the application output. If the application requires input, you can click in the Console window and type any value you need to type. Press Enter to enter the input, just as you would at the command line. When the application stops running, you’ll be able to work with the editor again. Let me know if you have any questions about opening, examining, and running book source using Eclipse at [email protected].

 

Working with Eclipse for Windows Developers (Part 1)

A number of my Windows readers have written to ask me about the Eclipse Integrated Development Environment (IDE). Part of the problem stems from the many versions of Eclipse. You can find a version of Eclipse for nearly any language out there. At least, it seems like there is a special version. What you really get is a plug-in for Eclipse Standard edition. Yes, if you’re working with C++, then getting the Eclipse IDE for C/C++ Developers makes sense. However, since I work with a lot of different languages, I simply download the standard edition and then tweak it as needed with the required plug-ins. The reality is that Eclipse is incredibly flexible.

 


When working with my books, all you need is the standard edition unless I specifically state otherwise in the book. Using one of the special editions can make Eclipse a little more difficult to use and that’s something you really don’t need when you’re just starting out with a new language. In addition, unless my book specifically says otherwise, you don’t need to download any of the plug-ins. All of the plug-ins are there to extend the functionality of Eclipse, which is nice when you need it, but potentially confusing when you don’t.

After you have selected the flavor of Eclipse that you want by clicking its link, you’ll find that Eclipse typically comes in 32-bit and 64-bit versions for Windows, Mac OS/X, and Linux developers. If you’re using a 64-bit version of Windows, you’ll want to download a 64-bit version of Eclipse to ensure you can make maximum use of the operating system features.

Selecting a platform displays another page where you select the site you want to use for downloading the product. Generally speaking, choosing one of the mirror sites will provide greater download speed than the main site, but your experience will likely vary from mine. What you get is a .ZIP file containing Eclipse. Of course, most Windows users are used to getting an executable of some sort with a complete installation program. All you really need to do is extract the Eclipse files anywhere on your hard drive to use it. I normally place Eclipse in its own folder off the main directory so that I don’t run into read/write permission problems inherent in putting it in the \Program Files folder.

At this point, you can start using Eclipse. That might seem odd, but there really isn’t anything more you need to do. Of course, opening Eclipse by finding its location in Windows Explorer every time you want to use it is a pain. Use these steps to make things easier.

 

  1. Right click the Eclipse.exe file in the host directory and choose Copy from the context menu.
  2. Right click Start | All Programs and choose Open from the context menu. You’ll see a copy of Windows Explorer open that has the appropriate folder selected.
  3. Select the Programs folder in the left pane to open it.
  4. Create a folder for Eclipse and open it.
  5. Right click anywhere in the Eclipse folder and choose Paste as Shortcut from the context menu. You now have an easy way to access Eclipse from the Start menu.
  6. (Optional) Rename the shortcut to something that’s a bit easier to read, like Eclipse.


At this point, you can right click the Eclipse entry and choose Pin to Taskbar from the context menu to make it even easier to access. However, that’s really all there is to installing Eclipse. Removal is even easier—just delete the shortcuts you’ve created, along with the Eclipse folder. Let me know if you encounter any problems getting Eclipse on your system at [email protected].

 

Thinking Through Indentation and IDE Automation

I’ve been asked a number of times about code indentation in my books by publishers, editors, and readers alike, so I thought it might be a good idea to talk about the matter in a blog post. It’s important to indent your code to help make the code more readable—to help make the flow of your code easier to see. Each indentation represents another code level. The application is at one level, classes at another, methods within the class at another, loops within a method at still another, and so on. By viewing the various levels of code within an application, you see an outline of application functionality and can better understand how each application element works. Because indentation is so important to the understanding of code, I spend considerable time working through indentation in my books so that you can better understand the examples I present.

Indentation is there for the developer to use as an aid to understanding. A few early computer languages, such as COBOL, were positional and depended on coding elements appearing in certain positions on each line. Most languages today don’t require that you use any sort of indentation. In fact, more than a few languages would let you write the entire application on a single line without any indentation at all (as long as you supplied a space between statements and expressions as needed). The compiler doesn’t care whether you indent using spaces or tabs, whether there are three or four spaces per indent, or whether you provide an indent for continued lines. The fact is that all of these characteristics are controlled by the developer to meet the developer’s needs.

Modern IDEs make it easy to indent your code as needed by performing the task automatically in some cases. For example, given an example of a Java method, you could begin by typing:

public void MyProc()
{

The moment you press Enter after the curly bracket (}), most IDEs will add an indented line and a closing curly bracket. Your cursor will end up on the indented line, ready for you to type a statement. You see something like this in your IDE:

public void MyProc()
{
 
}

The developer hasn’t done anything so far to provide indentations for the code, yet the code is already indented. Let’s say you create a variable like this:

boolean MyVar = true;

You then type a statement like this:

if (MyVar == true)

It’s at this point where some IDEs continue to provide an automatic indent and others don’t. For example, if I press Enter at the end of this line in Eclipse, the IDE automatically indents the next line for me. However, when I’m working with CodeBlocks, pressing Enter leaves the code at the same indention level as before. Neither approach is wrong or right—simply different. If I were to add a curly bracket on the next line to hold multiple lines of code within a block, CodeBlocks will already have the cursor in the right position and Eclipse will have to outdent the curly bracket to position it correctly.

Let’s say that this if statement has just one line of code following it, so I don’t use a curly bracket. When working with Eclipse, the cursor is already at the correct position and I simply type the next statement. However, when working with CodeBlocks, I must now press Tab in order to indent the line of code to show that this line (and only this line) is associated with the if statement. In this case, the example simply outputs a statement telling the value of MyVar like this.

System.out.println("MyVar == true");

Pressing Enter automatically outdents the line when using Eclipse because the next statement is automatically at the same level as the if statement. However, when working with CodeBlocks, I must press the Backspace to outdent the line manually. The resulting method could end up looking like this:

void MyProc()
{
    boolean MyVar = true;
    if (MyVar == true)
        System.out.println("MyVar == true");
     
}

Of course, a good developer will add some comments to this code as a reminder of what task the code performs and why the developer chose this particular technique to perform the task. The point of this post is that indentation is an essential part of working with most languages in order to make the resulting application easier to understand.

A final thought on IDEs is that most of them make it possible to configure the editor to indent or not indent to meet the requirements of your organization or personal tasks. IDEs commonly allow the use of tabs or spaces for indents (spaces are better when you want to write documentation). You can also choose the right amount of indentation (three spaces is optimal for books where space is limited). Let me know your thoughts on indentation and how you use it at [email protected].

 

Visual Studio 11 Updates

Microsoft plans to release the next update for Windows 8 during the first week in June. Most developers are also looking forward to an update of Visual Studio 11 about the same time. Visual Studio 11 contains a number of modest, but important, feature updates. As a result, I’ve been reading everything I can find on these new features and doing a little testing myself. So far, I haven’t seen much mention of the new debugging features or the new/updated tools provided with the new release. These changes are important nonetheless.

Of course, the most important of these updates is the ability to create Metro applications. Visual Studio 11 provides a complete set of templates you can use to create Metro applications using a combination of HTML5 and JavaScript. From the confusing assortment of posts that I’ve read, I’m not really clear as to whether the main download site provides you with a copy of Visual Studio 11 that includes full support for developing Metro applications. It turns out that you need the SDK in order to build these applications. In order to play with Metro applications, I downloaded the Visual Studio 11 beta from the Metro-style applications site, which definitely includes the SDK. This download only installs on a Windows 8 system. The Metro-style applications site also includes a number of other helpful downloads.

The feature that seems to be garnering the most attention though is the appearance of the new IDE. Many developers find the new IDE incredibly depressing to use. According to a number of sources, the beta team has heard the pleas of testers and decided to do something about it. Essentially, the changes are limited to a difference in colors. The updated IDE will be lighter gray and use some brighter colors in the icons. Of course, having a tool that’s fun, or at least interesting, to use is a requirement. No one wants to work with a depressingly dark gray tool all day. Still, I have to wonder why this particular feature is receiving so much press.

Of the features I’ve tried so far, I’m finding the C++ language additions the most tempting. For example, the IDE now makes it a lot easier to see various C++ elements through the use of color coding. A feature called Reference Highlighting is also interesting because it makes it easy to move between instances of a keyword within a source code file with greater ease. There is also built-in support now for the C++ 11 specification version of the Standard Template Library (STL). If you want, you can even build Metro-style applications using C++. The IDE also makes it a whole lot easier to work with code snippets. These new additions do make the IDE faster and more efficient, but also add complexity. Readers of C++ All-In-One Desk Reference For Dummies will be happy to know that I’ll continue using the GNU C++ compiler for the reasons stated in my Choosing the GNU C++ Compiler post.

Because I know I’ll eventually need to provide some level of Metro application support, I have been playing around with the Metro functionality. As part of my reading, I checked out the information on the Metro-Style Design Applications site. From a developer perspective, there are some pros and cons about these new requirements. For example, some developers see them as actually limiting application functionality and making applications less useful. I’m sure that what will happen is that developers will find new ways of adding functionality to applications that fit within the Microsoft guidelines and still offer a great application experience. Undoubtedly, Microsoft will also be tweaking those Metro design documents.

Have you done anything with Visual Studio 11? If so, let me know about the features you like best and which features you wish Microsoft would change. One of the most important questions for me is whether you see yourself using Visual Studio 11 for serious application development anytime soon. Knowing these answers will help me create better blog posts for you in the future. Contact me at [email protected].