New Location

As of September 7th, 2013 this blog will be moved to http://onehourhacks.com!

Simple Credit Card Validator

Validating credit cards numbers is a fairly simple job, and there are no short supply of algorithms out there you can easily download to do it. That being said, it is a perfect algorithm to write in Haxe for the following reasons:

• Speed isn't super critical
• It doesn't involve the system in any way
• You can use the same code across PHP, JS, Java, C++, C#, Actionscript etc.
• The same code would be useful in all contexts.

Algorithm

The algorithm realized here is the Luhn algorithm. You can do the formal proofs with some discrete math, or trust the Wikipedia Article that it is useful because:

• It detects any one digit error e.g. 1235 instead of 1234
• It detects any one transposition e.g. 1324 instead of 1234

Both errors are exceedingly common for human typists.
(Note that the ISBN10 algorithm holds some of these same properties.)

The Code

/**
A simple credit card validator that just runs the luhn algorithm against the 
given number. It doesn't even check to see if the string is long enough!
**/
class Validator
{
    /**
        Call this function with a string of digits that represent a credit card
        number.
    **/
    public static function validate(cardNumber : String) : Bool
    {
        return (luhn(cardNumber) == 0);
    }

    private static function luhn(numericString : String) : Int
    {
        var output = stringToDigitArray(numericString);
        var sum : Int = 0;
    
        var isEven : Bool = true;
        for( i in output.iterator())
        {
            if(isEven)
            {
                sum += sumOfDigits(i * 2);
            }
            else
            {
                sum += i;
            }
        
            isEven = !isEven; // Invert sign for next iteration
        }
    
        return sum % 10;
    }

    private static function stringToDigitArray(numericString : String) : Array<Int>
    {
        var output = new Array<Int>();
        for( i in 0...numericString.length)
        {
            output.push(Std.parseInt(numericString.charAt(i)));
        }
    
        return output;
    }

    private static function sumOfDigits(number : Int) : Int
    {
        var sum : Int = 0;
    
        while(number != 0)
        {
            sum += number % Std.int(10);
            number = Std.int(number / Std.int(10));
        }
    
        sum += number;
    
        return sum;
    }

    /* Examples */
    public static function main() : Void
    {
        // False
        trace(Validator.validate("1111111111111111"));
        
        // All true below this point
        trace(Validator.validate("4271058012370682"));
        trace(Validator.validate("4052047247466840"));
        trace(Validator.validate("4250112315345738"));
        trace(Validator.validate("4302220036446780"));
        trace(Validator.validate("4271382676044056"));
        trace(Validator.validate("4027883785016825"));
        trace(Validator.validate("4125875428334483"));
        trace(Validator.validate("4231380166327600"));
        trace(Validator.validate("4820020713741757"));
        trace(Validator.validate("4013112216053363"));
    }
}

HOWTO Find and Restore a Deleted SVN File/Folder

1. Change directories to your SVN project.
2. Run svn log -v > all_history.txt
3. Open all_history.txt and find the file you want (Ctrl + F is your friend!)
4. Make sure the line of the file you want begins with a D, this means it was deleted.
5. Look up above that file for the revision number, subtract one from it.
6. Run a new checkout for the version right beforehand i.e.: svn co http://hostname/path/to/files_parent_directory@revision#_right_before

Example

We want to find DataFeedList.java, here is the svn log:

------------------------------------------------------------------------
r24843 | joslewis | 2013-04-14 10:24:28 -0600 (Sun, 14 Apr 2013) | 1 line
Changed paths:
   D /trunk/software/src/bmod/gui/widgets/DataFeedList.java
   D /trunk/software/src/bmod/gui/widgets/FilterableComponentList.java
   D /trunk/software/src/bmod/gui/widgets/JValidatingTextField.java
   D /trunk/software/src/bmod/gui/widgets/WebButton.java
   D /trunk/software/src/bmod/gui/widgets/WebMenuItem.java
   M /trunk/software/src/bmod/plugin/generic/gui/ReportBugMenuItem.java

Modified ReportBugMenuItem to now work without WebButton. -~400 lines.
------------------------------------------------------------------------

We can see it was deleted (D) during revision r24843 by me on 2013-04-14 10:24:28 -0600. That means we want to actually get the version right before it was deleted, so we'll check out version 24842.

It was in the folder: /trunk/software/src/bmod/gui/widgets/, so we'll craft a new checkout with the information to get it back:

svn co http://svn.host.com/trunk/software/src/bmod/gui/widgets@24842

Run this in another folder so you don't overwrite the existing source, and you'll find the deleted item in it!

The Haxe Programming Language

This is a language that has seemed to slip by under the radar for the most part. Haxe, commonly pronounced "hex" or "hacks" is a language that targets many virtual machines, instead of the more common virtual machine that targets many languages.
It has a reasonable standard library that feels designed, much like Python's. The syntax is Go like, with identifiers following names:

class WebFetch 
{
    static function main()
    {
        var output = haxe.Http.requestUrl("http://www.google.com/");
        Sys.print(output);
    }
}

When built, this program is a ~68Kb JAR file when targeting Java or a 550Kb ELF when generating C++.
Compilation is straightforward, consisting of an internal build system that resolves libraries well, and a global repository that you can pull new packages from.
One of the biggest benefits of this language is its ability to compile and run on so many platforms; it easily handles:

• Android
• Flash
• Linux
• Mac
• Windows
• HTML + JS
• PHP
• Java (in Beta)

Example Uses

• You're a web-developer and have some business code that needs to target language of the day. Haxe can compile to language of the day without the need to translate.
• You want to code a cross-platform game to reach as broad of an audience as possible without re-writing lots of code.
• You want to develop a client for an API you publish so developers will actually use it without learning all of the languages it is going to be used in or maintaining the libraries.
• You want to develop an app that is future proof, but don't mind it being a little ugly.

Pitfalls

• There is no standardized UI system.
• The coding conventions are a strange to begin with.
• There are some obvious deficiencies in the standard library (no string compare)
• Code is slightly larger than it would be natively.

Conclusion

Overall, Haxe is a good language that I see myself using in the long run because it is so future-proof and well designed. Plus, the ability to do things easily cannot be underestimated. The example above would have taken over 100 lines of Java and about the same in C++, instead it took eight and thirty seconds to put together after knowing the language less than twenty-four hours.

Finding Strings in Raw Data

Occasionally, you'll find yourself wishing you could pull out human-readable strings from binary formats, like JPEGs, EXEs or even raw disk dumps. This may be for a variety of reasons, such as:

• Performing malware analysis
• Finding lost files
• Efficient searching

Algorithm

Actually extracting interesting text from a file is somewhat difficult because,

• you don't know what format the file is in
• you don't know what language the text is in
• you don't know how relevant a snippet is

Therefore, we can only rely on if a character is printable or non-printable, whatever that means. Then we can use information to try and weed out the junk without compromising things like IP addresses, which may look very much like junk, but are not.

The method I found that works relatively well is this:

1. Look for runs of printable characters in a file with length greater than 5
2. Look for runs of uppercase letters, lowercase letters, or numbers.
3. For each character of each run, set the value of the nth letter to n.
4. Sum the run totals
5. Sort the scores and return.

This algorithm is implemented in the code below.

Outputs

The software below strips special terminal characters (like newlines and carriage returns), and prints out items in the following format:

<score><tab><file><tab><string><newline>

Example:

00000250    dd.img  &'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz
00000176    dd.img  .IEC 61966-2.1 Default RGB colour space - sRGB
00000176    dd.img  .IEC 61966-2.1 Default RGB colour space - sRGB
00000138    dd.img  ;CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 95 
00000136    dd.img  ,Reference Viewing Condition in IEC61966-2.1
00000136    dd.img  ,Reference Viewing Condition in IEC61966-2.1
00000099    dd.img  Copyright (c) 1998 Hewlett-Packard Company
00000078    dd.img  <?xpacket begin='
00000075    dd.img  Hhttp://ns.adobe.com/xap/1.0/
00000075    dd.img  Adobe Photoshop CS2 Macintosh
00000063    dd.img  $54545,,4,44455,554,,4,,,4,55456,,,,,,/5554-,,4,24,
00000058    dd.img  IEC http://www.iec.ch
00000058    dd.img  IEC http://www.iec.ch
00000051    dd.img  bottomOutsetlong
00000046    dd.img  rightOutsetlong

It appears this unallocated space actually has files!

Operation

Save the below code to reader.py and provide it paths to the files you want it to read. There are no parameters.

Source

#!/usr/bin/env python
'''
A program that extracts strings from binary files.

Usage: reader.py file [file ...]

Copyright (c) 2013, Joseph Lewis III <joseph@josephlewis.net> | <joehms22@gmail.com>
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

    Redistributions of source code must retain the above copyright notice, this 
    list of conditions and the following disclaimer.
    
    Redistributions in binary form must reproduce the above copyright notice, 
    this list of conditions and the following disclaimer in the documentation 
    and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

'''

import sys

ALPHABET_UPPER = frozenset("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
ALPHABET = frozenset("abcdefghijklmnopqrstuvwxyz,-. \t\n")
NUMBERS = frozenset("0123456789")
OTHER_CHARACTERS = frozenset("!\"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~ \t\n\r")
PRINTABLE = frozenset('0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~ \t\n\r')
READ_BLOCK_SIZE_B = 2048

def type_of_char(c):
    if c in ALPHABET_UPPER:
        return 'A'
    if c in ALPHABET:
        return 'a'
    if c in NUMBERS:
        return 'n'
    return 'o'

def value_of_string(string):
    ''' A heuristic to give an approximate value to a string. '''

    # make sure the string isn't total junk
    if all([c in OTHER_CHARACTERS for c in string]):
        return 0

    types = [type_of_char(c) for c in string]
    runs = [0 for c in string]

    # higher runs are better
    for i in range(1, len(types)):
        if types[i] == 'o':
            runs[i] = 0
        elif types[i] == types[i - 1]:
            runs[i] = runs[i - 1] + 1
        else:
            runs[i] = 0

    return sum(runs)

def process_string(string, extracts):
    if len(string) < 5:
        return

    score = value_of_string(string)

    if score > 0:
        extracts.append((score, path, string))

def extract_strings(path, extracts):
    '''Extracts "interesting" strings from a file'''

    mystr = []

    with open(path, 'rb') as fd:
        chars = fd.read(READ_BLOCK_SIZE_B)

        while chars != "":
            for char in chars:
                if char in "\r\n":
                    mystr.append(" ")
                elif char in PRINTABLE:
                    mystr.append(char)
                elif len(mystr) == 0:
                    continue
                else:
                    process_string("".join(mystr), extracts)
                    mystr = []
            chars = fd.read(READ_BLOCK_SIZE_B)

    process_string("".join(mystr), extracts)


if __name__ == "__main__":
    if len(sys.argv) == 1:
        print("Usage: {} file [file ...]".format(sys.argv[0]))
        exit(1)

    # a list of tuples (score, document, string)
    extracts = []

    for path in sys.argv[1:]:
        extract_strings(path, extracts)

    extracts = sorted(extracts, reverse=True)
    for score, path, string in extracts:
        print("{:08.0f}\t{}\t{}".format(score, path, string))

Friday Operating System News

Operating Systems

NixOS is a neat little OS that is built around a single config file. The file can be moved to a different computer, and it'll set itself up as a clone, or as you're more likely to use it, as a single file that does version control on your system (so next time you upgrade, you can roll it back right away!)

Jolla has released it's first phones running Sailfish OS.

Finding Deleted JPEGs in Disk Dumps

If you've ever accidentally deleted items from your digital camera, or want to find them from a drive that has been lost, this script could be for you!

It works by looking through a disk image for a sequence of bytes that signify the beginning and end of a JPEG image.

Running It

First save the below script as ImageCarver.java

Then compile it:

javac ImageCarver.java

Afterwards, run it with the path to the disk image you want it to search:

java ImageCarver disk.img

It will report images it found, as well as the locations in the disk that it found them:

1.jpg 0xe3fa 0x10272
2.jpg 0xe000 0x20865
3.jpg 0x1000 0x2557e
4.jpg 0x41000 0x4437d
5.jpg 0x26000 0x654d4
6.jpg 0x25000 0x96816
7.jpg 0x86000 0x9eca0
8.jpg 0x6d000 0xdc614
9.jpg 0xe2000 0xe7ac9
10.jpg 0xa4000 0xeb28a
11.jpg 0xed000 0xfc9a9
12.jpg 0x136000 0x13b08e
13.jpg 0x1b5144 0x1b629e
14.jpg 0x1b5000 0x1b794a
15.jpg 0x1b67f0 0x1bea75
16.jpg 0x1c9000 0x1cfada
17.jpg 0x22d000 0x237824
18.jpg 0x281000 0x288928

The Script

import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.SeekableByteChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.util.LinkedList;

/**
 * Carves images from a file given from the command line.
 * 
 * @author Joseph Lewis <joehms22@gmail.com>
 *
 */
public class ImageCarver 
{
    byte[] STARTING_BYTES = new byte[]{(byte)0xFF, (byte)0xD8,(byte)0xFF};
    byte[] ENDING_BYTES = new byte[]{(byte)0xFF,(byte)0xD9};
    int imgno = 0;


    /**
     * Carves images out of a file.
     * @param sbc - the byte channel to read from.
     * @throws IOException - on a file error
     */
    public ImageCarver(SeekableByteChannel sbc) throws IOException
    {
        // find all potential starting and ending points in the file.
        LinkedList<Long> startingPoints = ringMatch(sbc, STARTING_BYTES);
        LinkedList<Long> endingPoints = ringMatch(sbc, ENDING_BYTES);

        // keep writing out we're out of images
        while(startingPoints.size() > 0)
        {
            carveImage(sbc, startingPoints, endingPoints);
        }
    }

    /**
     * Carves an image out of the file.
     * @param sbc - the channel to read from
     * @param starts - the start positions of the image
     * @param ends - the end positions of the image
     * @throws IOException - on error
     */
    public void carveImage(SeekableByteChannel sbc, LinkedList<Long> starts, LinkedList<Long> ends) throws IOException
    {
        // clear off no longer useful end points
        while(starts.peek() > ends.peek())
        {
            ends.pop();
        }
    
        // If ends is empty, add the literal file end.
        if(ends.size() == 0)
        {
            ends.push(sbc.size());
        }
    
        // get our starting position.
        long sp = starts.pop();
    
        // Check to see if there's an image inside of us.
        if(starts.size() > 0 && starts.peek() < ends.peek())
        {
            carveImage(sbc, starts, ends);
        }
    
        // get our closest end point.
        long endloc = ends.pop();
    
        imgno++;
        createImage(imgno + ".jpg", sbc, sp, endloc);
    }

    /**
     * Extracts an image from the SeekableByteChannel.
     * @param name - the image name
     * @param src - the channel to read from
     * @param begin - the offset to the beginning of the image
     * @param end - the offset to the end of the image
     * @throws IOException - on error
     */
    public void createImage(String name, SeekableByteChannel src, long begin, long end) throws IOException
    {
        // alert the user about the image we're outputting
        System.out.println(String.format("%s 0x%x 0x%x", name, begin, end));
    
        // write out the given slice of bytes
        try(OutputStream os = Files.newOutputStream(Paths.get(name), StandardOpenOption.CREATE))
        {
            src.position(begin);
            ByteBuffer b = ByteBuffer.allocate((int) (end - begin));
            src.read(b);
            os.write(b.array());
        }
    }

    /**
     * Searches through a channel looking for a given array of bytes to match.
     * 
     * @param sbc - the channel to read from
     * @param toMatch - the bytes to match in the stream
     * @return a list of all offsets where the given bytes are found
     * @throws IOException - on file read error
     */
    public LinkedList<Long> ringMatch(SeekableByteChannel sbc, byte[] toMatch) throws IOException
    {
        // holds all found vars
        LinkedList<Long> finds = new LinkedList<Long>();
    
        for(long i = 0; i < sbc.size() - toMatch.length; i++)
        {
            ByteBuffer b = ByteBuffer.allocate(toMatch.length);
            sbc.position(i);
            sbc.read(b);
        
            boolean matches = true;
            for(int j = 0; j < toMatch.length; j++)
            {
                if(b.array()[j] != toMatch[j])
                {
                    matches = false;
                    break;
                }
            }
        
            if(matches)
            {
                finds.add(i);
            }
        
        }
        return finds;
    }

    public static void main(String[] args) throws Exception 
    {
        // Make sure we have an arg.
        if(args.length != 1)
        {
            System.out.println("Must supply exactly one argument.");
            return;
        }
    
        Path p = Paths.get(args[0]);
    
        try (SeekableByteChannel sbc = Files.newByteChannel(p)) 
        {
            new ImageCarver(sbc);
        }
        catch (IOException x) 
        {
            System.out.println("Caught exception: " + x);
        }
    }
}

Doing a Diff on a Specific Git Version

Sometimes you just need to look up something that changed between versions of a repository instead of reverting a full file, here two git commands come in handy.

The first shows the log of your recent commits, along with hex numbers which represent the versions:

git log

When you find the revision you want, just type:

git show af78ec673f46db23ae5537a8dbea1253e6e844f3

replacing the revision number with the one you want, and you'll get a diff from that version!

If A Robot Makes Your Makefile

If a robot makes your Makefile, it is no better than pressing that magic green arrow at the top of an IDE and watching the project compile.

LIST=CPU
ifndef RECURSE
RECURSE=recurse.mk
ifdef CONFIG
RDIR=$(dir $(CONFIG))
endif
endif
include settings.mk
include $(RDIR)$(RECURSE)

While a robot may easily understand this, it isn't any better than using g++ by hand.

Makefiles, in this programmer's humble opinion, should be clean, elegant, and easy to understand. They are the easiest way to immediately jump in to a project and figure out what is important and the way it is structured.
Even more importantly, they can point to dead ends and code that is never actually accessed by the program, keeping wasted days at bay.

A Simple Python Plugin Framework

This framework is somewhat different than those currently out there, most of those either:

• don't allow dynamic plugins
• require you to use nasty hooks
• require specifically formatted imports
• require plugins to all be in one place

This script however allows plugins to:

• be dynamically loaded
• be a subclass of a master plugin class
• have any name
• be in many different folders

The general idea is that you give the function a path, and the base-class that all plugins are subclasses of, then it iterates through the path, finding python files, loading them, and then searching them for any classes that are subclasses of the class you want. When it gets done, it returns them:

Code

import inspect
import os
import sys
def load_plugins(plugin_path, instance):
    '''Loads a set of plugins at the given path.

    Arguments:
    plugin_path - the OS path to look for plugins at.
    instance - classes of this instance will be returned
    '''
    plugins = []
    plugin_dir = os.path.realpath(plugin_path)
    sys.path.append(plugin_dir)

    for f in os.listdir(plugin_dir):
        if f.endswith(".py"):
            name = f[:-3]
        elif f.endswith(".pyc"):
            name = f[:-4]
        elif os.path.isdir(os.path.join(plugin_dir, f)):
            name = f
        else:
            continue

        try:
            mod = __import__(name, globals(), locals(), [], 0)

            for piece in inspect.getmembers(mod):
                if isinstance(piece, instance):
                    plugins.append(piece)

        except ImportError as e:
            print(e)
            pass # problem importing

    return plugins

Friday Link List -- Ubuntu 13.04 Backend Updates and Good News for the Web

Real Dev News for Ubuntu 13.04

This is way better than the stuff you'll find on OMGUbuntu!

1. gvfs updates with MTP Support (you can now connect your Android 4 device and it just works)
2. LibreOffice 4.0; a huge number of bugfixes went in to this
3. cups updates to now auto-detect shared printers on your network, and auto-share them with other devices including iPads/iPhones etc.
4. Much of the backend has been ported to Python 3 in hopes of totally removing Python 2.
5. Lots of work has gone in to sandboxing applications from one another (most likely due to the phone release)
6. A potential new feature (if it makes it in time) is automatic removal of old kernels, which can take up a few gigs of space over the course of six months if not cleaned manually.
7. New upstart version allowing userspace (beta) jobs, and launch on file/folder change, hopefully meaning fewer background services and less clunky code.
8. Fixes for quite a few RAM hogs that persist across sessions.

Web

• jQuery 2.0 has been released it drops support for IE 6-8, it looks like old Android (2x) is going to get the chop next.
• Chrome forked WebKit to form Blink, they say they were able to drop about 2.5 million lines of code right off the bat, hopefully it'll speed up their iteration time.
• Firefox has released it's baseline compiler tl;dr version, it cuts down on complexity, and does a much better job of supporting IonMonkey being it's based on the same backend giving a huge speed boost.
• ASM.js promises a good future for porting C/C++ apps to the browser with support from Firefox, and Chrome in the works.

Things I Learned From Working A 2 Year 50,000 Line Project

This is mostly to serve as a brain dump for a piece of software I've been writing for around two and a half years, it was comprised of a large database (read: ~five hundred million rows), a cross platform Java application, a large Python web-server/API, a phone app, lots of small scripts that fetched and transformed data, and globs of glue. (I wrote all of them in the two year period!)

Architecture

• It pays to fully think through your APIs first
• All web APIs should have a version in the URL e.g. /application/api/2.0/...
• Profile early and often
• A few good caches are worth many bad ones
• Make everything a plugin that extends one class; you may think it's a big hairy mess, but it's better than multiple plugin systems.
• Use libraries wherever possible
• Ditch libraries when you only use one function from them

Debugging

• Users write awful bug reports
• You can help them write better ones by giving them a short form to fill out.
• Automatically attach debugging info to all bug reports
• Make version numbers easily available (not just in the executable's name)
• Auto-update your apps so you don't have users lagging behind

Upgrades

• Your requirements will constantly change, make sure you have an easy upgrade path in your database that won't force hacky solutions.
• Make sure your ORM supports incremental changes.

User Experience

• Start fast, even if you're not done processing yet
• Never trust an engineer to do design (including software engineers)
• If the user doesn't see it, it doesn't exist
• Make it very hard for the user to do stupid things
• Do automatic backups/saves

Sanity

• Clients like saying "I did say that, didn't I? What I meant was..."
  • all features should be plugins so you can enable/disable when they do this
• SVN and Eclipse are a nightmare to get playing nice
• Java 7 is really awesome!
  • auto-closing streams
  • multiple catches in a try block
  • string in switch statements
• JNLP sucks, but so does embedding apps in-browser; just make it easy for the user to download it, and provide auto upgrading

Covertly Sending Messages Using DNS Queries

I won't be the first to post about this, but I read it a while back and thought it sounded fun: note, this post will be more technical in nature than most, but I hope it will be a fun read, and a fun toy.

Let's say you want to send short messages that are very covert and not persistent. You could, of course use Twitter, sending strange cryptographic tweets, but that may be too obvious for your tastes, plus they can more easily be traced back to an IP/Person if just one slip-up is made. Somehow using DNS would be an awesome way to do this because:

• It can be accessed by anyone
• Messages expire over known intervals
• No accounts are necessary
• There are huge amounts of data to go through and junk requests that will look just like yours if someone wanted to find you out
• Google has a very nice public DNS you can use for free at 8.8.8.8 or 8.8.4.4

Specifics

The two types of queries we'll be using here are standard and non-recursive DNS queries.

Standard queries are the ones that your computer normally does to change a hostname in to an IP address it can contact, to do this from a command line you'd type:

$ dig @8.8.8.8 www.google.com

To which the DNS server responds:

; <<>> DiG 9.8.1-P1 <<>> @8.8.8.8 www.google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5459
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.google.com.            IN  A

;; ANSWER SECTION:
www.google.com.     300 IN  A   74.125.225.179
www.google.com.     300 IN  A   74.125.225.180
www.google.com.     300 IN  A   74.125.225.176
www.google.com.     300 IN  A   74.125.225.177
www.google.com.     300 IN  A   74.125.225.178

;; Query time: 53 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 12 10:18:31 2013
;; MSG SIZE  rcvd: 112

This shows you all of the numerical IP addresses you can reach google.com at, if you copy and paste the first one in to your browser, you'll see Google's home page.

That's all well and good, but how would you actually store information using DNS? For this you'll need to generate a domain name that doesn't exist, we'll be using www.testdnsflagsetting.com for the purposes of this article:

Now do a dig on it:

$ dig @8.8.8.8 www.testdnsflagsetting.com +norecurse

The response looks different this time, note that ANSWER: 0 and AUTHORITY: 0, this means the request is not authoratative (8.8.8.8 doesn't know this informatoin for sure), and there was no answer.

This is because we passed in the +norecurse flag to dig, asking the server to just reply with what was it had stored rather than asking higher up in the chain of command:

; <<>> DiG 9.8.1-P1 <<>> @8.8.8.8 www.testdnsflagsetting.com +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23556
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.testdnsflagsetting.com.    IN  A

;; Query time: 57 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 12 10:04:10 2013
;; MSG SIZE  rcvd: 44

Now we'll do the same thing, but without the +norecursive:

$ dig @8.8.8.8 www.testdnsflagsetting.com

; <<>> DiG 9.8.1-P1 <<>> @8.8.8.8 www.testdnsflagsetting.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 22889
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.testdnsflagsetting.com.    IN  A

;; AUTHORITY SECTION:
com.            900 IN  SOA a.gtld-servers.net. nstld.verisign-grs.com. 1365782647 1800 900 604800 86400

;; Query time: 82 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 12 10:04:21 2013
;; MSG SIZE  rcvd: 117

Note that the AUTHORITY: 1 now is true, meaning that 8.8.8.8 checked with the server that knows about all .com addresses to see if ours existed. There is also a new section, note the number 900 in it, that means this response is going to be saved in Google's DNS (8.8.8.8) for 900 seconds.

If we go back and do the +norecurse version again, we now get the cached response, but with 890, meaning there are only 890 seconds left until the cache is cleared:

$ dig @8.8.8.8 www.testdnsflagsetting.com +norecurse

; <<>> DiG 9.8.1-P1 <<>> @8.8.8.8 www.testdnsflagsetting.com +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 14832
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.testdnsflagsetting.com.    IN  A

;; AUTHORITY SECTION:
com.            890 IN  SOA a.gtld-servers.net. nstld.verisign-grs.com. 1365782647 1800 900 604800 86400

;; Query time: 39 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 12 10:04:31 2013
;; MSG SIZE  rcvd: 117

Put these together now, and say you want to send a true or a false to someone on the other side of the globe, you both just have to agree on a time to check, and a domain, then if you want to say true you do a recursive query, and the other person will get back an authoritative record when they do a non-recursive; if you don't do the query, they will not and they'll know you sent false

Sending a Whole Message

Now say you want to send a whole message! Convert the text to ASCII, and break it down to bits, use the first byte to tell how long the message will be (up to 255 characters).

Say you just wanted to send the message Hi, that means you'd send a 2 then Hi, but you can't use the same domain the whole way through, you'll have to agree on a bunch of domains, or a way of randomly generating them. Here we'll just append a number to the end:

Letter:       2        H        i
Binary:   00000010 01001000 01101001
POsition:            111111 11112222
          01234567 89012345 67890123

Your queries would look something like this:

dig www.testdnsflagsetting6.com
dig www.testdnsflagsetting9.com
dig www.testdnsflagsetting12.com
dig www.testdnsflagsetting17.com
dig www.testdnsflagsetting18.com
dig www.testdnsflagsetting20.com
dig www.testdnsflagsetting23.com

You only actually dig for the 1s, because they are represented by true.

Getting The Message Back Out

Now to get the message back out, you just need to do a dig for positions 0-7, and convert that in to the number of letters in the message, multiply that by 8 to get the number of bits you need to check, and check the domains ending with all of those numbers.

First 8

dig www.testdnsflagsetting0.com

dig www.testdnsflagsetting1.com

...

dig www.testdnsflagsetting7.com

This would give us the number 2, meaning there were 2 more sections of 8, then you read on.

A program to do it!

This is a short Python script to do it, you may want to change the DOMAIN_NAME. in case other people are using the script too.

#!/usr/bin/env python3
''' A program to send messages via DNS queries.
Copyright 2013 Joseph Lewis <joehms22@gmail.com> | <joseph@josephlewis.net>

MIT License
'''

import subprocess

DNS_SERVER = '8.8.8.8'
DOMAIN_NAME = "www.testdnsflagsetting{}.com"
NORECURSE_OPT = "+norecurse"

msg = raw_input("Enter a message, or blank to receive: ")

def read_byte(byteno):
    byte = "0b"
    for i in range(byteno * 8, (byteno + 1) * 8):
        output = subprocess.check_output(['dig','@{}'.format(DNS_SERVER), DOMAIN_NAME.format(i), NORECURSE_OPT]) 
        if ";; AUTHORITY SECTION:" in output:
            byte += '1'
        else:
            byte += '0'

    return int(byte, 2) # converts binary to an int

def write_byte(byteno, byte):
    to_write = bin(byte)[2:].zfill(8) # gets binary representation of a byte
    for loc, b in enumerate(to_write):
        if b == '1':
            i = (byteno * 8) + loc
            subprocess.check_output(['dig','@{}'.format(DNS_SERVER), DOMAIN_NAME.format(i)])
            print "Wrote 1 at: {}".format(i)

if len(msg) == 0:
    message = ""
    for byte in range(1,read_byte(0) + 1): # first byte is length of message
        message += chr(read_byte(byte))

    if len(message) > 0:
        print message
    else:
        print "[No Message]"

else:
    total = len(msg)
    write_byte(0, total)
    for loc, char in enumerate(msg):
        write_byte(loc + 1, ord(char))

    print "Message written"

HOWTO Not Worry About IE6 Ever Again

While IE6 market share is dwindling, we are far from being out of the woods yet. Gecko, Trident, WebKit and Blink, are likely to never convirge on a full implementation of W3C specifications, leaving some browsers more equal than others.

Not to worry though, there is a nice little script called Modernizr, that can auto-detect features of a browser and automatically load fixes for browsers that don't have the technologies you need.

Shortlist of Features Modernizr Can Detect

• HTML5 Canvas
• IndexedDB
• WebWorkers
• Lots of CSS3 Stuff
• Geolocation
• WebSockets

Coding is a Drug

Coding is highly addictive. I suspect it has to do with the combination of the brain being the bottleneck, like all good puzzles; rapid iteration allowing you to see a result slowly emerge, and the fact that once you get done, you'll have something tangable as a reward that will serve you for a long time.

It's somewhat like combining crossword puzzles, woodworking, and sketching.

HowTo Calculate an MD5 hash in Java

If you're dealing with networking, or just want to make sure your files don't get corrupted/modified between times your app runs.

It is critical to use the algorithm.reset() command before using a MessageDigest instance, as if there are bytes still remaining from a prior digest, it will completely ruin your output.

Code

import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

// calculates an MD5sum for a given bit of text
public class MD5 
{
    public static void main(String[] args)
    {
        try 
        {
            MessageDigest algorithm = MessageDigest.getInstance("MD5");
            algorithm.reset(); // clear the digest back to starting state.

            // Get selected digest of the given text.
            byte[] digest = algorithm.digest( "Hello, world!".getBytes() );

            System.out.println(byteArrToHex(digest));
        } 
        catch (NoSuchAlgorithmException e) 
        {
            // no such algorithm, MD5
        }
    }

    // converts bytes to text.
    private static String byteArrToHex(byte[] bytes)
    {
        StringBuilder sb = new StringBuilder();
        for(byte b : bytes) 
            sb.append(String.format("%02X", b)); 
        return sb.toString().toLowerCase();
    }
}

Other Notes

• To calculate SHA1 hashes instead, use SHA1 as the instance type.
• The NoSuchAlgorithmException likely won't be thrown on desktop platforms, but it is possible that certain algorithms won't be implemented in certain areas due to them being considered military grade encryption; SHA1 and MD5 shouldn't have this problem because they are just simple hashes.

The Four Keys To Good Design

The points here are stolen from The Design of Everyday Things which you should definitely purchase from the publisher.

There are four major design aspects that all interfaces should have, many already exist in the physical world due to physical processes, but need to be replicated in the visual world for users to feel in control and comfortable.

Visibility

At all times, the user should be able to quickly see what is going on and what options they have.

• Change icons when a process is running 
• Give timers estimating how much time is left 
• Gray out options that can't be chosen 
• Give context-sensitive menus for a given context one of the best I've seen here is the Adobe Flash Designer whose entire bottom toolbox changed based upon what you had selected, yet it was still predictable.

A Good Conceptual Model

This is one of the biggest issues with design, especially in complex systems that could never be made in the physical world.

Email is a nice example, you open an "Inbox" and read "Mail" that was "Sent" to you. Of course, you don't want to tell your users that email is mail, otherwise you end up with the dreaded "Please email me back the file, I sent you the only copy I have".

But the abstraction does remove doubt from the user's mind that the process is stable and will work, even if it doesn't work in the way that they believe or you portray.

Good Mappings

Every action should have an immediate and visible response, if the user presses "Send", don't wait on the message screen until the message is sent to return the user to the Inbox. Show the user what happened whenever they do something so they can build relations in their mind about what that is doing in software.

Feedback

It doesn't matter so much that your software does computation, it just needs to inform the user of that or keep the user occupied while it is doing it; much in the same way that Mac OSX 10.6 shows an intro video: All new operating system installs need to do routine checks and get themselves in order, but there is surely a better way than this:

HOWTO Quickly Create PHP Forms with phpMyEdit

One of the biggest hassles of building a site in PHP is that there is no way to easily generate forms for your database. The same is true for most systems, CRUD mechanisms take most of the design time, even when the core logic is simple.

In PHP there is a simple program called phpMyEdit that takes the time out of creating these forms. Unfortunately, it is rather old and only works with MySQL, but for most projects that is what you need.

To Setup

1. Download phpMyEdit
2. Extract the contents in to the directory of your site.
3. Go to: http://example.com/phpMyEdit-5.1/phpMyEditSetup.php replacing 5.1 with your version of phpMyEdit.
4. Enter your database credentials and server.
5. Choose your database.
6. Choose your fields.
7. Create the form!

You'll want to secure the forms, of course, behdind something that checks for administrator rights, but that is about it!

Hacking Respondus LockDown Browser 2.0

It has been about a year since I've visited this topic, and I've decided that it is indeed worth revisiting. But first, a few reasons that I'm doing this so nobody gets the wrong idea:

• To prove Respondus LockDown Browser does not circumvent cheating
• To encourage anyone left using this technology to change their assessment methodology to other means, such as projects because:
  • They are more effective
  • It is just as easy to determine if a student cheated
  • They test ability to apply instead of memorization (life is open book, what you need to know is how to apply what you know!)
• To boost blog viewership (I write to be read, and Respondus is one of my most read posts)

The Technique

I have devised a better way of "hacking" Respondus that does not involve the use of monitoring system calls, although that would be equally viable and was what I had originally intended to do.
This version works better, is cleaner, and anyone can edit it when it stops working. Essentially, it just modifies Respondus' window and opens other programs with button presses.

How To Run

1. Download the code, and save it to a file on your desktop called Respondus.ahk
2. Download and install Auto Hot Key
3. Close all other windows on your desktop.
4. Right click the Respondus.ahk file and choose "Run As Administrator"

Code tested on Windows 8/XP. You might need to change the first line in 64 bit versions of Windows 7/Vista to reflect another path to LockDown.exe.

Code

Run, C:\Program Files\Respondus LockDown Browser\LockDown.exe

WinWait, Respondus LockDown Browser
WinSet, AlwaysOnTop, Off, Respondus LockDown Browser
WinSet, Enable, , Respondus LockDown Browser


Gui, Add, Button, default, &Show IE
Gui, Add, Button, default, &Hide Respondus
Gui, Add, Button, default, &Show Respondus
Gui, Add, Button, default, &Maximize All
Gui, Show,, Subversion Menu

WinSet, AlwaysOnTop, On, Subversion Menu
return

ButtonShowIE:
Run, IEXPLORE.EXE http://onehourhacks.blogspot.com
WinWait, One Hour Hacks - Windows Internet Explorer
WinMove, One Hour Hacks - Windows Internet Explorer, , (A_ScreenWidth/2), 0, (A_ScreenWidth/2), (A_ScreenHeight),,
return

ButtonHideRespondus:
WinHide, Respondus LockDown Browser
return

ButtonMaximizeAll:
WinGet, WindowList, List
Loop, %WindowList%
    {
    WinMaximize, % "ahk_id " . WindowList%A_Index%
    }

WinHide, Respondus LockDown Browser
return

ButtonShowRespondus:
WinShow, Respondus LockDown Browser
return

ButtonLeftSideRespondus:
WinMove, Respondus LockDown Browser, , 0, 0, (A_ScreenWidth/2), (A_ScreenHeight),,
return

ButtonFullScreenRespondus:
WinMove, Respondus LockDown Browser, , 0, 0, (A_ScreenWidth), (A_ScreenHeight),,
return

GuiClose:
ExitApp

The Case Against Customizability

Three words on building customizability in to your product: don't do it. This is to the products that allow users to change colors, organize panels, choose icons, etc.

Users have a hard enough time with finding functionality

Watch your technophobic family members try to attach an image to an email. While you may have no problem getting through menus to do it, they often do. Remember: there are more of them than us.
Customizability just gets in the way for super-users because they have better ways of doing it.

Your Product is a Tool, Not a Piece of Art

Your goal as a designer should be to minimize the amount of time on your software because it should just work.
Get your users back to the real world where they can go create art or enjoy it.

Troubleshooting/Documenting is Hard With Customizability

When you write tutorials, it is easy when everything is the same. Customizability makes it harder to follow tutorials, especially if the writer has a custom setup.
Users will also find it harder to help one another, which means more support is needed from you!

It Leads to Messy Code

Just think about rearranging components on the web, you need a place to save the information, a new server interface for submitting it, security procedures over that to make sure it isn't an attack vector, another library user-side to test in every browser.

Most of the Users Will Never Do It

Take BlackBoard for example, I TA a bit, and of all the students I've seen pull up blackboard, not one of them has changed the style or layout as they're allowed to; nobody wants to spend the time on a product they use twice a day.

Your time is better spent on usability

iGoogle was a great example of this, instead of needing a homepage like they used to, users can now just type what they want in to the search box and get information relevant to them.
Spend your time figuring out why users would need to customize, and eliminating that need.

Final Note

I'm not advocating that you throw everything in to code, no no! I fully support MVC, that is how superusers can change your environment to suit them best. If you must allow user tweaks to your software, like moving around menus, put it in a plugin so when your code gets updated, the whole software doesn't break.