Local SVN Repositories

We all know that SVN is an incredibly handy tool while developing.  Remember that brilliant idea you had at 3 a.m. last night that you implemented and it completely screwed up the program?  SVN does.

Sadly getting SVN repositories is, quite frankly, a hassle when your program is small and you don't want to share it with the world yet.  Google Code takes ten or fifteen minutes to set up, Sourceforge takes a few days for your project to be approved, and they don't work on weekends.  Then what about checking out when you are out of range of the Internet?

I do know that setting up a dedicated server for your personal SVN is a pain, plus you will need to back up the server on a regular schedule, keep it connected to the Internet, secured, and working through your NAT/Firewall, and pay for the electricity it uses.

Materials:

• Subversion 

Procedure:

1. Install Subversion sudo apt-get install subversion
2. Make a new folder for the subversion repositories in your home directory, I named mine svn_repository , remember explicit is better than implicit.
3. Use the svnadmin command to create a new repository for your project like so: svnadmin create ~/svn_repository/my_project
4. Now you can check out your project using the file path as the URL.  svn co file:///home/username/svn_repository/my_project ~/my_project_working_copy

Make a GUI wrapper for your favorite CLI program.

Quick, what is your favorite Posix command line program?  What do you mean you don't have one?

In this tutorial I'll show you how to write quick and dirty Python GUIs to wrap some of those programs you use often, but not often enough to remember all of the arguments.  I will be using the apropos command to demonstrate.


If you try to stay out of the command line interfaces as much as possible it's very understandable, was it -r or -R for recursive?  Oftentimes a gui is very comforting because you can see all of the options displayed very spatially rather than in a long string of confusing arguments.  This spacial layout slows the process down for anyone that really knows what they are doing, but honestly is that you?  I use man pages and --help commands extensively while working with unfamiliar commands, which is slower than a GUI.

Materials:

• Your favorite Linux/Unix distro
• Python
• WX Python
• WX Glade

Procedure:

1. Download and install Python, wxPython, and wxGlade.  For me that was sudo apt-get install python-wxversion python-wxglade , (Ubuntu 10.10)
2. Start up wxGlade, and design a gui around your application, for me (apropos) I needed a search box, and a results box, with some fancy labels to point out which was which.



General Layout






Program Preview













3. Rather than having the user click a button when they want to search, we will have apropos search each time the text box is changed, this is a waste of system resources, but apropos is fast and I like instantaneous results.  So add a listener to your tiny list box in the Events tab of the Properties box when you select it.  Because you want this event called any time new text is entered put a method name in the EVT_TEXT handler box, this method will be called when new text is entered. 



Adding A Method




4. Select Application in the Tree, your properties box will now change to the export options.  Choose Python, and add an output path for the generated code. Then press Generate Code.



The Code Generation Box

5. You can now run your newly created Python program, it will have the shape of an application but no functionality yet.
6. Open the python file in your favorite text editor, and replace the method that you called in EVT_TEXT with one that you make.  Mine looks like this:

def enter_search_text(self, event): # wxGlade: MainFrame.<event_handler>
       
        import subprocess
        #Call an os function with each arg a separate list element
        process = subprocess.Popen(["apropos", self.text_ctrl_1.GetValue()], stdout=subprocess.PIPE)
        output, unused_err = process.communicate()
        #clear output
        self.text_ctrl_2.Clear()
        #Add the results to the output text control
        self.text_ctrl_2.AppendText(output)
            
        event.Skip()

7. You could have named your text controls in wxGlade, I chose not to due to laziness.
8. Close the program out and run it, you will now be able to search the man pages for programs and settings available on your system.  If I wanted to find all of the calculators on my system I would type in calculator and apropos would find them.

xcalc looks interesting

Conclusions:
Building a GUI for CLI programs is a great way to learn more about the programs themselves, while making life easier for yourself.  You also get to discover new programs and learn Python GUI programming.

Java Source Code Scraping

I am taking a course in Java development whose textbook provides example programs.  Unlike many other books the source code for these programs is trapped inside of HTML pages.  Being a programmer I don't like doing work, and therefore spent some time making a nice Python script that will take in an URL, and folder path, and will scrape the given page for source code, remove all formatting, and output them properly named to the folder specified.



With a little work this script could scrape whatever you wanted, as long as it is between certain tags.

#!/usr/bin/python

import re
import urllib2
import xml.sax.saxutils

numnones = 0

def unescape(string):
    '''Unescape the & escapes in html, like " returns a string '''
    
    string = string.replace(""", "\"")  #Not done by xml lib
    string = xml.sax.saxutils.unescape(string)
    
    return string
    
def remove_HTML_tags(text):
    '''Removes html tags from a supplied string.
    
    Warning: This is accomplished using regular expressions that simply cut 
    out all text in between and including less than and greater than 
    characters.
    
    '''
    
    regex_html = re.compile(r'<.*?>')
    return regex_html.sub('', text)

def fetch_page(url):
    '''Returns the html for the webpage at the supplied url.'''
    
    page = urllib2.urlopen(url)
    pagetext = ""
    
    for line in page:
        pagetext += line
    
    return pagetext
    
def get_name(class_text):
    ''' Returns the name of the abstract, class, or interface, when given the
    text of a Java file.
    
    Returns None if the name can not be determined.
    
    Warning: Although Java classes may be named with unicode, this function 
    will only return characters A-Z, a-z, and 0-9.
    
    '''
    
    #Compile the regular expressions, they all ignore case and accept all chars.
    class_ = re.compile(r'public class (.*?) ', re.DOTALL | re.IGNORECASE)
    interface = re.compile(r'public interface (.*?) ', re.DOTALL | re.IGNORECASE)
    abstract = re.compile(r'public abstract class (.*?) ', re.DOTALL | re.IGNORECASE)
    
    #Find the name of the class/interface/abstract
    name =  class_.findall(class_text)  #Returns [] if none found

    if name == []:
        name = abstract.findall(class_text)
    
    if name == []:
        name = interface.findall(class_text)
    
    #If no name is found return None.
    if name == []:  
        return None 
        
    #Remove any remaining non a-z A-Z characters
    accepted_chars = re.compile(r'[^A-Za-z0-9]*')
    return accepted_chars.sub('', name[0])
    
    
def return_between(first_tag, second_tag, text):
    '''Returns an array of the text between the delimiters given. All text 
    between the end and beginning delimiters will be discarded.
    
    Arguments:
    first_tag -- The tag to begin text harvesting. (string)
    second_tag -- The tag to end text harvesting.  (string)
    text -- The string in which the tags are to be found. (string)
    
    '''
    
    basic_split = text.split(first_tag)
    
    #select only the sections which contain the close tags, discard the rest.
    second_split = []
    for i in basic_split:
        if second_tag in i:
            second_split.append(i)
    
    #Cut out the text before the close tag
    between = []
    
    for line in second_split:
        value, end = line.split(second_tag, 1)
        between.append(value)
        
    return between
    
def write_file(name, content, location, ext=".java"):
    '''Writes a file with the name given, with the content provided, to the
    location given, with the given extension.
    
    Arguments:
    name -- The name of the file to be written.
    content -- The content of the file to be written.
    location -- The folder where the file will be created/overwritten.
    
    Keyword Arguments:
    ext -- The extension for the file. (default ".java")
    
    Warning: This function will overwrite any pre-existing files without 
    giving warnings.
    
    '''
    
    if not location.endswith("/"):  #FIXME This is not cross platform!
        location = location + "/"
        
    print("Location: "+location)
        
    f = open( location + name + ext, "w" )
    f.write( content )
    f.close()
    
def main(url, output_folder):
    '''Fetches the text at the supplied url and outputs .java files of code
    between pre tags.
    
    Arguments:
    url -- The url of the page to scrape code from. (string)
    output_folder -- The directory that the java files will be created in.
    
    '''
    global numnones
    page_text = fetch_page(url)
    
    #Get the text between the pre tags
    code_and_garbage = return_between("<pre>", "</pre>", page_text)
    
    #Walk through each entry clean it, and write the source file
    for i in code_and_garbage:
        
        print i
        
        i = remove_HTML_tags(i)
        i = unescape(i)
        
        name = get_name(i)
        
        if name == None:
            write_file("None"+str(numnones), i, output_folder)
            numnones += 1
        else:
            write_file(name, i, output_folder)

if __name__ == "__main__":
    location = raw_input("Enter the url: ")
    output_folder = raw_input("Enter the folder to write to: ")
    
    main( location, output_folder )

Parallel Port Flasher

How to turn that old computer in to a robot.

Today, I decided to resurrect an old Toshiba laptop that was sitting in my drawer, the processor is an ancient Pentium III 450mhz, 320mb of RAM, and a 4.9Gb hard drive, even phones put this laptop to shame.  The good thing with these kinds of systems is that you don't mind messing them up.  It took a little more than an hour to get this all working.

Materials:

• Old computer with parallel port you don't mind playing with.
• DSL Live CD
• Parapin
• An LED
• Wires
• Alligator Clips
• 2 Paperclips

Procedure:
The Software Part:

1. Start up your computer with the DSL Live CD.
2. I installed DSL, because this system was built for Windows 98, and DSL works much better anyhow.
3. Open the MyDSL Browser and under System install gcc1-with-libs.  Finding this made me incredibly happy as it contained all of the stuff I needed.
4. Download and extract the Parapin source. (I used 1.5.1-beta1)  The folder may not show up unless you right click in the file browser window and press update.
5. Open that directory and run setup-2.4.sh, being that DSL runs a 2.4 series kernel.
6. Build the library using make .
7. Create the file pptest.c in the parapin directory with the following contents:

#include "parapin.h"
#include "unistd.h"

int main()
{
    /*Init the first parallel port, change to LPT2 to 
    use the second */
    pin_init_user(LPT1); 

    /*These pins can either be inputs or outputs*/
    pin_output_mode(LP_PIN02); 
    
    while( 1 )
    {
        set_pin(LP_PIN02); /*Turn on the pin*/
        sleep(2);
        clear_pin(LP_PIN02); /*Turn off the pin*/
        sleep(2);
    }

    return 0;
}

8. Now run gcc pptest.c parapin.c this compiles the program.

The Hardware Part

1. Unfold two small paperclips so they look like 6's, this should take one bend.
2. Place their tips in hole 2, and 25 of the parallel port.  The larger row has 13 pinholes, this will be the top, they are numbered from 1-13 sequentially from the right to the left.  The smaller row (bottom) is numbered 14-25 right to left.
3. Connect the anode of the LED (long side) to the paperclip in hole 2, and the cathode to the paperclip in hole 25.  I did this with four alligator clips and some wire.

Final Steps

Within the directory you compiled the software should be an executable named "a.out".  You need to execute this as sudo, if you are working in the directory you compiled in just enter "sudo ./a.out".  Sudo is needed because this program accesses hardware.  Your LED should begin to turn on and off every two seconds.

In an early test I connected a battery tester to the leads and watched needle jump rather funnily between on and off states, this can't be seen using the LED.

Conclusion

While it would take (me) days to do anything useful in c with the parallel port I/O, parapin looks like it has Python and Java libraries, so with a few relays, lights, and switches you could make a robot from that old computer.

DSL also provides some nice daemons like SSH, and HTTP, if you got wireless working on your machine, you could create an ad-hoc network and control your robot over the air!

One rather curious thing I noticed was that if the program was quit while the LED was in an on state, the light continued to glow.  Perhaps this is in the actual parallel port specification, or an artifact of the library or OS.  There also seemed to be a slight discharge when I shut the system down, probably from some capacitor finally being allowed to discharge.