Thursday, March 31, 2011

The File Server

I was recently tasked with sending a friend a file the other day.  It was too large for email, he was on Windows and I on Ubuntu, so we couldn't netcat it, and the network we were on disallowed SMB (windows share) traffic.  So I pulled out my trusty USB drive, and gave him the file.  Honestly though, I don't want strange USB drives being stuck in my computer.

Solution:  Build a python web-server that can upload and download files :)


A while back I built a CGI Python webserver by hand, this is not the story of that, but rather of my personal file server.

First things first, I needed a way to upload files, index.py would work:

self.send_200()
self.wfile.write('''
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Upload/Download</title>
<style TYPE="text/css">
body {
background-color:#ADD8E6;
text-align:center;
margin:0px;
padding:0px;
font-family:airal,helvetica;
}
.main{
display:block;
background-color:#E5E5E5;
margin-left:auto;
margin-right:auto;
padding:10px;
width:800px;
}

</style>

<script type="text/javascript">
function getNameFromPath(strFilepath) {
var objRE = new RegExp(/([^\\/\\\\]+)$/);
var strName = objRE.exec(strFilepath);

if (strName == null) {
return null;
}
else {
return strName[0];
}
}

function setName()
{
document.getElementById('filename').value = getNameFromPath(document.getElementById('up').value);
}
</script>
</head>

<body>
<div class="main">
<H1>Upload / Download Files</H1>
<form action="upload.py" enctype="multipart/form-data" method="post">
<input type="file" name="myfile" size="chars" id="up">
<input name="filename" id="filename" type="hidden"></input>
<input type="submit" value="Send" onclick="setName()">
</form>


<hr>
<h2>Download</h2>
<ul>
''')

import os
j = os.listdir("Downloads")
for item in j:
self.wfile.write("<li><a href='download.py?f=%s'>%s</a></li>" % (item,item))


self.wfile.write('''
</ul>

</div>
</body>
</html>''')

Essentially all of the HTML is echoed back to the user, however there is also a list of previously uploaded files that needed to be available for download, so that was auto-generated from a directory (not web accessible).

Next I needed the upload script to actually save files that people uplodad:
if POST_DICT:
try:
print POST_DICT.keys()
print POST_DICT['filename']
path = os.path.join("Downloads", POST_DICT['filename'])
print path

with open(path, 'wb') as f:
f.write(POST_DICT['myfile'])
except Exception, e:
print e

self.redirect("index.py")

This script checks to see if the POST variable is set, if so it saves the file as the filename (remember to do it binary otherwise we could get a mess when non text files are uploaded). Then redirect the user back to the index so they can instantly see their newly uploaded file.

Download.py
The last, and most difficult part was the download script:
import os
import mimetypes

if QUERY_DICT:
filename = QUERY_DICT['f'][0]

mime = mimetypes.guess_type(filename)

path = os.path.join("Downloads", filename)
head={'Content-Disposition':'attachment; filename="%s"' %(filename)}
self.send_200(mime_type=mime, headers=head)
with open(path) as f:
self.wfile.write(f.read())

else:
self.redirect("index.py")

This was because unless the content-type was set properly, and the proper filename was given, a file would be saved to the drive every time named "download.py".

Finished Product

Saturday, March 19, 2011

The Polygraph Machine

A few months ago I was rummaging though a box and found some electronics components.  To immediately digress I can never throw such things away, and when old electronics are being tossed, I'm there with a soldering iron to rescue parts.  Anyhow, I had found a small amplification chip, a headphone jack and some DIY PCB.  A long time ago, the specifications for the OLPC XO-1 laptop had a curious feature, they were going to use the microphone in port as something other than just a microphone, Geiger counters, conductivity checkers, and a whole slew of things that wouldn't have needed a driver that were just basic I/O systems.


A very simple polygraph just registers the resistance of skin (you can see where I'm going now, can't you?)  By itself the microphone port wouldn't have had enough power to check conductivity, but if you used a battery and an amplifier, well, that is another story.

Around midnight, after drawing a quick schematic (and I'm sure an improper one), and having a poison scare dealing with Ferric Chloride (accidental ingestion), I came up with this:

The Polygraph


To finish off here, I installed Audacity hoping that it would work rather than building some PyGame/Matplotlib/Python monstrosity for a simple test of the equipment.  Some scary sounds came in when I placed my fingers on the terminal.  Attached here is the first sample I took, some features to note:
  • High buzzing sound
  • Second 27?
  • Seconds 21-24, Cannon in D?

 



Friday, March 4, 2011

Gnome 3 - The red-headed stepchild.

Yesterday I grabbed a Gnome 3 livecd from their website.  While the project may not have the fanfare of Ubuntu, or even the excitement that Elementary brings to the table, it was the most beautiful OS I've used in years.

When I began using Ubuntu about four years ago I was amazed, everything seemed to be right were I would have put it, the interface was remarkably ugly though, yet it was free, stable, and ran great on my old Toshiba.  Gnome 3 takes the usability of Ubuntu and makes it top notch.

Some quickly noticeable things about the UI made it very intuitive:
  • There is no need to click the menu, just bump the corner which makes it blindingly fast.
  • The clock is right in the middle, making it significantly easier for your eyes to find the time than when the clock was in with the jumble of icons.
  • The gnome-panel is dark and the windows are light, drawing your focus away from the panel, it disappears in your vision making the whole thing application-centric.
  • Workspaces are incredibly easy to use and I might actually use them now.
  • Opening applications and documents can be done from the keyboard much easier than now, the super key is actually used for something important.
  • Windows don't have a minimize or maximize button (windows don't need to be minimized to the tray due to the fast window and workspace switcher) and maximize functions can be performed by double clicking the title bar, or by an aero-snap type functionality.  The lack of buttons makes the whole thing clean.
  • The few panel menus that exist are clean and consistent behaving in the same way and providing very useful menus.
  • The control panel is finally integrated and looks amazing.
  • There is an easy way to mark applications as trusted (chmod +x) from the GUI.
Downsides:
  • It runs from a livecd, but doesn't like my nvidia graphics card when installed to the hard-drive (after the first time)  The same goes for Ubuntu 11.04.
  • There were no icons on the desktop (perhaps the desktop doesn't support them?)  It was therefore hard to find the "Install to disk" .desktop file as it was on the desktop.

Monday, February 28, 2011

DDoS Repercussions


Over the past few months law enforcement and rogue groups have been battling over the Wikileaks controversy in an on-line battlefield. The players are the same as those of the crypto-wars, albeit a few generations younger on both sides.  Recently the entire thing ended in a raid of forty suspected DDoSers' homes.  But were those raids justified?  To what extent should you expect privacy on-line, and exactly why are both of these groups that are promoting liberty doing it in such different manners?

Wikileaks Timeline:

For those of you that have been living under a rock for the past few months, here is a quick time-line of events:

  • November 28, 2010 - Wikileaks releases a quarter of a million diplomatic cables. That day a DDoS attack is conducted upon the site causing it to shut down.
  • November 29, 2010 - Wikileaks moves to Amazon's EC2 cloud.
  • December 1, 2010 - Amazon drops Wikileaks under supposed pressure from the U.S. Government.
  • December 2, 2010 - Wikileaks loses its DNS records.
  • December 8/9, 2010 - DDoS attacks are conducted upon Mastercard and Visa sites through "Operation Payback" by Anonymous.
  • January 27, 2011 - The Federal Bureau of Investigation paired with international law enforcement conducts searches of forty suspected DDoS perpetrators (surprisingly absent are warrants for the original DDoS attack on Wikileaks). 

    What is DDoS?



    To begin with the legality of raids, we must examine what DDoS attacks actually are.  Essentially a Distributed Denial of Service attack, is a flood of requests to a server, when a server receives such a flood it slows down considerably or can even crash; these attacks have been around for years and can have quite simple defenses.  What makes DDoS attacks so powerful is that on one side, you have a few servers, and on the other thousands of individuals (or infected hosts in a botnet) to distribute the load of connections, so the attacker's Internet doesn't slow down in the process.  These attacks used a fairly simple tool called the Low Orbit Ion Cannon (LOIC) which essentially conducts these attacks automagically.  What makes it so powerful is that script kiddies/"hacktivists" can easily download it and fire it up (easier than metasploit, sqlmap or firesheep).  A major drawback when using the LOIC is that it dosen't mask the originating IP of the attackers (this plays a key role later on).

    Denial of Service attacks have been likened to different things depending on which side of the law one resides on.  For attackers denial of service attacks can be likened to ringing a doorbell and running away, except it can be done hundreds of times a second.  For law enforcement such attacks constitute criminal offenses under Title 18 U.S.C. § 1030, specifically because the computers "attacked" were "protected computers" being that they belonged to a financial institution in the United States.


    Tooting the Same Horn:
    What is most confusing about the DOS attacks and the raids is that both sides argue that they are promoting freedom.  From Anonymous' viewpoint, companies that attempted to stop the flow of stolen diplomatic cables, (as they were undeniably stolen), were restricting the freedom of the individual's right to know about their government's proceedings.  From the alternative viewpoint, by tracking the users of the LOIC, law enforcement was able to determine sources of the attack and stop them.  There are a few problems with the way they decided to do this however, cheifly:
    • Some of the people whose doors were broken in didn't participate in the attack (IP addresses change, and there are multiple users per IP).
    • No individual caused the sites to experience service outages, rather a collective did.  The same thing happens to twitter quite often, too many people start tweeting and cause outages, the publicity of these outages oftentimes make them worse.
    • The publicity these sites received during the attack drove people to "see what was happening" causing a snowball effect.
    • The (inept) security firm that aided in the capture of the suspected criminal IP addresses used legally questionable social engineering type searches to find the reported ringleaders.
    Being that security and freedom really is key, what should be done in the future to mitigate these problems?
    Part of the problem of the Internet, along with the best thing about it, is that it is dynamic.  If I wanted to commit a crime, I could just patch myself in anywhere (coffee shop, McDonalds, my neighbor's wireless etc.) and begin attacking; within an hour I could be gone and could have left their fingerprints all over the mess.  Either the government needs to be able to trace packets as they are traveling, or better yet ISPs just need to drop the traffic that their clients ask them to.  If doors are going to be knocked [on | down] then the government needs to make sure that it isn't some kid running a free piece of software on their computer, or my grandma that got infected with a bot.
    The statement listed above by the FBI said that the individuals could win a free ten year trip to prison along with significant civil liability.  We all know that this is like music sharing, if the government went after every DDoS participant they would be in more of a financial mess than they are now.  There seems to need to be some middle ground here, a fifty dollar ticket would deter more people than ten years in prison, because they know tickets are easier to do than taking everyone to a drawn out legal procedure.
    As far as snowball goes, perhaps a media campaign could fix that, say ISP, DDoS, IP, SYN ACK, and TCP to your neighbor and their eyes will glaze over; we know not to visit a site under attack, but they don't.  To them a computer is a magic device that does stupid things for some reason, (to us it is an electric-impulse physics god that we screw up by doing stupid things).  To them Google is just out there *points to the sky*, rather than on a rack server at the other end of a screaming fast trans-continental fiber optic line; the public just needs to know that they shouldn't check things out for themselves.
    What Anonymous did was like civil disobedience, although someone in there had a nefarious intent.  That does not excuse the use of social engineering to deconstruct the organization.  Social engineering is a dangerous practice as anyone can easily mimic another person to use as a decoy, and there is no way to really know on-line (proxies/onion routing/etc.).  It is moronic to believe that social-engineering against social-engineers is sensible; especially when you are using silly methods like the law-enforcement contractor did.  Now that these methods are out in the open, we'll probably see an influx of spoofed half-true seemingly linked accounts that in actuality are run by different people.
      While we may be over these particular raids right now, and not much can be done about them, what happened was none-the-less wrong.  What kind of a future do you want, one in which everyone is "safe" from attacks (that come from US individuals, rather than bot-nets or foreign entities), or one where civil disobedience comes at almost no price (other than social backlash) to the corporations being "attacked"?

      Saturday, February 26, 2011

      XKCD Fetcher

      Problem:  You want every XKCD in existence on your harddrive.  Solution: Pirate this script!

      Instead of going through and raping the XKCD website, they have provided a nice JSON document for every comic.  The script below can produce two outputs, one is a nice list of comics that includes number, title, and alt-text, that looks like this:



      • 405 - Journal 3 - Oh, and, uh, if the Russian government asks, that submarine was always there.
      • 406 - Venting - P.P.S. I can kill you with my brain.
      • 407 - Cheap GPS - In lieu of mapping software, I once wrote a Perl program which, given a USB GPS receiver and a destination, printed 'LEFT' 'RIGHT' OR 'STRAIGHT' based on my heading.
      • 408 - Overqualified - To anyone I've taken on a terrible date, this is retroactively my cover story.
      • 409 - Electric Skateboard (Double Comic) - Unsafe vehicles, hills, and philosophy go hand in hand.
      • 410 - Math Paper - That's nothing. I once lost my genetics, rocketry, and stripping licenses in a single incident.

      The other will just print a list of URLs to stdout, that you could write to a file or possibly pipe to a wget script.  To use this function just invoke the program with a -w option.

      One caveat, this script will only fetch comics until the second number in the range statement, so change it before running. This script is licensed under an Apache/GPL/MIT/Other OSI approved license, just be sure to attribute me, thanks :) Remember that XKCD comics are CCbyNC so you are free to copy them to your heart's content.

      #!/usr/bin/env python
      #Copyright 2011 Joseph Lewis <joehms22 gmail com>
      #MIT License/GPL 2+ License/Apache License

      import urllib2
      import time
      import sys

      WGET = False
      try:
      if sys.argv[1] == "-w":
      WGET = True
      except IndexError:
      pass

      for i in range(1, 850):
      if i == 404: #Skip the 404th comic which is an error 404 page.
      continue
      url = "http://xkcd.com/%i/info.0.json" % (int(i))
      comic_json = urllib2.urlopen(url)
      time.sleep(.2)
      comic = eval( comic_json.read() )

      number = comic['num']
      title = comic['title']
      image = comic['img']
      alt = comic['alt']

      if WGET: #Just list urls for wget :)
      print(image)
      else:
      print("<li><a href='%s'>%s - %s</a> - %s</li>\n" % (image, number, title, alt))

      TI BASIC

      I just dug out my old TI-83 Plus and was poking around the PRGM menu and found a few old programs that I had made.  Most of them were little helper programs that I created while the teacher was lecturing in my tenth grade pre-calculus class (I didn't take calculus until three years later, by then most of it was gone).

      I had learned BASIC two years prior to learning TIBASIC, in an old math textbook that a teacher of mine had collected.  My original programs were written on a sheet of lined paper, and did simple things like solve triangles.  I was unable to find an suitable interpreter as BASIC had taken its leave years prior, I was simply born about ten years too late.

      A freshman in that class (who was a genius and later became a dear friend of mine), was very patient in teaching me the essentials of the language, I started off with a few simple things such as making the program distributed by the teacher to solve quadratic equations run twice as fast.  I also made a minesweeper, but that seems to have disappeared off the calculator.  Later on I also made PONG, but this too has gone (PONG ran very slowly, but none-the-less worked).  There was another game I made that made you race around the screen attempting to collect items...

      Anyhow that is enough reminiscing for today :)

      Friday, February 25, 2011

      Modeling The St. Petersburg Paradox

      Background:

      The St. Petersburg game is a game of chance; the person placing the wager is asked for a certain amount. After paying the casino to play a fair coin is flipped until a tails is reached. The player wins 2number of flips dollars. The chances of winning the nth round are given by 1/2^n for example the chances of winning the eighth round are 1/256 and the payoff for that round is 256. The expected payoff for any particular round is the chance of winning multiplied by the payoff, for this game it is always 1. The expected payoff for a game is the sum of all possible rounds. In this case there are an infinite number of rounds so the expected payoff is infinity.  Hypothetically then, the perfectly rational person will pay any amount in order to play, but how realistic is this?
      Setup:
      I decided to write a simulation in Python (of course) to actually check what the average player should pay to play.
      #!/usr/bin/python
      '''stpetersburg.py -- A saint petersburg simulator.

      Copyright 2011 Joseph Lewis <joehms22@gmail.com>

      MIT License http://www.opensource.org/licenses/mit-license.php

      '''
      import random

      def play():
      total_won = 0
      payoff_matrix = {}

      print("Iter:\tFlips:\tWinnings:\tWon?")

      for i in range(iterations):
      winnings = 0
      flips = flip(max_game) #Flip the coin until tails.

      #Determine winnings based upon the wager multiplier and flips.
      if flips:
      winnings = multiplier**(flips)

      #Stats
      total_won += winnings
      if flips in payoff_matrix.keys():
      payoff_matrix[flips] = payoff_matrix[flips] + 1
      else:
      payoff_matrix[flips] = 1

      print("%i\t%i\t%i\t\t%s" % (i, flips, winnings, winnings >= initial_wager))

      #Do some basic stats.
      avg_won = float(total_won) / iterations
      print ("Avg. Winnings:\t%d" % (avg_won))
      print ("Flips x Frequency")
      for k in payoff_matrix.keys():
      print("%s x %s" % (k, payoff_matrix[k]))
      print ("Money Won: %s" % (total_won))
      print ("Money Waged: %s" % (iterations * initial_wager))


      def flip(max_flip):
      '''Flips a coin until max_flip is reached or the coin turns up
      tails. Returns the number of flips that came up heads.

      If max_flip is -1 flipping will continue until a tails is reached.
      '''
      numflips = 0

      while numflips != max_flip:
      numflips += 1
      if random.randint(0,1): #0 = heads 1 = Tails:
      return numflips


      return numflips


      def ask_int(question, default):
      '''Asks the user for an int printing out the default and question.
      '''
      num = raw_input(question + " (Default: %s)\n" % (str(default)))
      if num == '':
      return default
      return int(num)


      if __name__ == "__main__":
      print __doc__ #Notify the user of the license (top of file)

      #Set up simulation
      iterations = ask_int("How many iterations?", 100000)
      initial_wager = ask_int("What is the initial wager?", 2)
      multiplier = ask_int("What is the multiplier per win?", 2)
      max_game = ask_int("What is the bound on the number of games -1 for infinity?", -1)

      #Start simulation
      play()
      Running:
      This is an output of the default settings (which can be tweaked to easily accommodate other versions of the problem).
      Avg. Winnings:  18
      Flips x Frequency
      1 x 50078
      2 x 25019
      3 x 12474
      4 x 6286
      5 x 3085
      6 x 1455
      7 x 791
      8 x 396
      9 x 197
      10 x 110
      11 x 51
      12 x 32
      13 x 17
      14 x 5
      15 x 2
      16 x 1
      18 x 1
      Money Won: 1858488
      Money Waged: 200000

      The output was fairly consistent the several times I have run it, the amount won is about ten times the money wagered, so therefore you should want to bet no more than about twenty dollars.  There you have it, a rational cutoff for playing the paradox :)