Fallout 3 -- Crash Fixed

I've been swearing at Bethesda Softworks for some time since my Fallout 3 stopped working for no apparent reason, when I upgraded from Vista x64 to Windows 7 x64.

It failed once before and it turned out to be an nVidia video driver update, and a new update fixed it. However it's been over 6 months now and multiple nVidia updates have failed to fix it.

I spent some time this weekend debugging it, and discovered it's some kind of issue with Fallout 3 and DisplayLink drivers. I had tried unplugging my Mimo 7" USB auxillary monitor before, and Fallout 3 had still crashed. It turns out, that it was still crashing inside Fallout 3 when it returned from the DisplayLink drivers. I assume there's some monitor iteration or some other call going on, and it's not happy when something comes back with bounds smaller than the main screen.

Anyhow, if you have issues with Fallout 3 and you have "other" video/monitor drivers, try uninstalling them.

Iterative Collatz in Python (with resume)

I'm assumming there will be a lot of people looking to play with Collatz soon.

Here's a non-recursing version that will resume when you restart it.
Given all numbers to 2^58 have already been checked you might be there a while...

It's trivial to get it to dump a CSV with the steps for graphing, but, this one doesn't do that (but gives you the steps so that you can).

If you don't know what this is for, it doesn't matter.. (but you can google it...)

import os
def collatz(n):
 steps = []
 stAppend = steps.append
 while n > 1:
  stAppend(n)
  
  while not (n & 0x1):
   n >>= 1
   stAppend(n)

  if n == 1:
   break
  n = n * 3 + 1
 return steps

if __name__ == '__main__':
 n = 2
 if os.access('collatz.prg', os.F_OK|os.R_OK|os.W_OK):
  n = int(open('collatz.prg', 'rb').read())

 while True:
  steps = collatz(n)
  open('collatz.prg', 'wb').write(str(n))
  print n, len(steps)
  n += 1

Adding a "submit on enter" in ExtJS formpanels.

I had to retrospectively setup "submit on enter" to some existing ExtJS forms.
Googling only turned up some fairly unappetizing solutions.
Here's a way using defaults. Just change the getCmp or parametize it if you want, I've left it as a constant for illustration purposes.
Add this to your FormPanel constructor and all your fields will get "submit-on-enter".
If you only want 1 field to have it then just add everything inside the defaults to that 1 field.

defaults:{
  enableKeyEvents:true,
  listeners:{
    specialKey: function(field, el)
    {
      if(el.getKey() == Ext.EventObject.ENTER)
      {
        Ext.getCmp('address-search-button').fireEvent('click');
      }
    }
  }
}

Decode Encrypted CueCat Output With Ugly Python

import sys; from string import maketrans as m; from binascii import a2b_uu as d
q = lambda x,y: ''.join([chr(c) for c in range(ord(x), ord(y) + 1)])
t = m(q('a','z') + q('A','Z') + q('0','9') + '+-', q(' ','_'))
def u(st): return ''.join([chr(ord(c) ^ 67) for c in d(chr(32 + 3 * len(st) / 4) + st)])
print "ID: %s, Type: %s, Barcode: %s"%(tuple([u(b.translate(t)) for b in sys.stdin.read().split('.')[1:-1]]))

12 Bar Blues on an Arduino

Using a small 8ohm analogue speaker hooked up to pin 11, by strobing the pin, I generate a square wave (a kind of annoying timbre to be honest).
I generate 2 octaves of semitones starting with A-220 and index into it.
The timing is currently inaccurate as the note length for lower notes runs slightly longer (half a pulse width) than higher notes due to how the strobing currently works. It's surprisingly noticeable. I need to work on that to make it more consistent. Probably by using timer interrupts instead of delays.

To do:

Alter the volume by using analogWrite() to set the duty cycle of the pin... 255 = loud 0 = off.
I think I can use that mechanism to generate other waveforms, I'll probably try sawtooth next.
Proper ADSR waveforms.
Multichannel output (haw haw).

const int baseFreq = 220; // A
const int numHalfTones = 24; // 2 Octaves
long frequencies[numHalfTones];
long halfFreqs[numHalfTones];

/* 12 semi-tones per octave
  | Bf| | | C#| Ef| | |F# |G# |
 -+---+ | +---+---+ | +---+---+-
| A | B | C | D | E | F | G | A |
+---+---+---+---+---+---+---+---+
 
 2 octaves: 
 0  1  2  3  4  5  6  7  8  9 10 11
 A Bf  B  C C#  D Ef  E  F F#  G G# 
 12 13 14 15 16 17 18 19 20 21 22 23
 */

int seq[] = {
  3,  7, 10, 12, 13, 12, 10 , 7,
  3,  7, 10, 12, 13, 12, 10 , 7,
  8, 12, 15, 17, 18, 17, 15, 12,
  3,  7, 10, 12, 13, 12, 10 , 7,
  10, 14, 17, -1,  8, 12, 15, -1
};

int notes = 40;

long getTimeHigh(int i)
{
  return halfFreqs[i]; 
};

void ToneGen(void)
{
  float baseMult;
  int i;
  int lastTone = baseFreq;  
  baseMult = pow(2.0 , 0.083333);
  for(i = 0; i < numHalfTones; ++i)
  {
    long frequency = baseFreq;
    if(i)
    {
      frequency = (0.5 + (frequencies[i - 1] * baseMult));
    }
    frequencies[i] = frequency;
    halfFreqs[i] = (0.5 + (1000000.0 / frequency / 2.0));
  } 
};

void setup(void) {
  pinMode(11, OUTPUT);
  ToneGen();
  Serial.begin(57600);
}

int rest_count = 100;
long duration = (60 * 1000000) / 288;

void playNote(int idx)
{
  long tone = 0;
  if (idx >= 0)
  {
    tone = getTimeHigh(idx);
  }

  long endTime = micros() + duration ;
  if (tone > 0) 
  { 
    int val = HIGH;  
    while (micros() < endTime) 
    {
      digitalWrite(11,val);
      delayMicroseconds(tone);
      val = !val;
    } 

  }
  else 
  {
    delay(duration / 1000);
  }   
  digitalWrite(11, LOW); 
}

void loop(void) {
  analogWrite(11, 0);
  for(int i = 0; i < notes; ++i)
  {  
    playNote(seq[i]); 
    delay(10);
  }
}

Steampunk LCD Monitor

To go with the steampunk keyboard and mouse, the guys over at steampunk workshop have made an absolutely amazing steampunk shell for and LCD-Flat panel.



You can see it being made here here.

How to scan in 3d using milk, lego, and a webcam.

This instructable shows how to create 3d models using milk, lego and a webcam. I love low-tech solutions, and this ranks right up there.

This video shows how it works;

Sendfile Patch for Zope2

I've put together a patch for Zope that allows code to use the sendfile system call to transfer data out of Zope.

As this requires a file object, it's only available for use from trusted code, so that means inside a Product or an External method.

The good news is, that it's fairly easy to write an External method that does something useful like serve contents from a directory.

Here's an example external method that serves files from '/shared/images'

import os
directory = '/shared/images'

def Image(self, filename):
    newPath = os.path.join(directory, filename)
    if not os.access(newPath, os.F_OK|os.R_OK):
        self.REQUEST.RESPONSE.setStatus(404)
        return ''
    f = open(newPath, 'rb')
    self.REQUEST.RESPONSE.sendfile(f, content_type='image/jpeg', other_headers = {'Content-Disposition', 'inline;filename=%s'%(filename))
    f.close()
    return ''

It uses the sendfile wrapper module from here.

It gives you a speed up of 2-3 times serving content out of Zope from the filesystem.
I also have a patch for ExternalFile to let this use the sendfile patch.

You can get the code and the patch from zope.org

Dick Cheney vs George Orwell

In an article in the Sydney Morning Herald titled Terrorists have ambitions of empire, says Cheney Dick Cheney is reported to have said;

[Terrorists' ultimate aim is to establish] "a caliphate covering a region from Spain, across North Africa, through the Middle East and South Asia, all the way to Indonesia -and it wouldn't stop there."

So this would mean we would see a map almost like this one (courtesty of wikipedia) where the yellow zone would indicate this caliphate;


The problem is, the yellow zones represent the site of the "Perpetual War" in George Orwell's 1984. In 1984 the perpetual war is unwinnable (and maybe fabricated), and is one of the mechanisms to to keep the population under control.

Amusing isn't it, that an Orwelling dystopia may be upon us?

ZopeTrac Updated

I have updated ZopeTrac to 0.10.3, which makes it work with Trac 0.10.3.

You can get it from here; here.

I will try to mimic the Trac release numbers so you can tell what version of Trac it last worked with.

Some Advertisers are stupid...

Whilst doing a search on how to lodge a false advertising complaint, the following advertisement appeared. Perhaps these people need to be more selective in the terms they want to advertise with.

Dumb.

Set Windows Wallpaper from Python

*** For multiple monitors see this post instead Set windows wallpaper from python for multiple monitors  ***


Here's some code to set a random wallpaper from a list of images, or a list of directories or a single image. It doesn't recurse subdirectories, but, that's just as simple to add if you really require that functionality. It requires PIL.

You can set it to change the wallpaper to rotate to a new one at a specified number of minutes.

It can use any image that PIL can read (it converts it to a BMP).

# Code to rotate or set wallpaper under windows
# Copyright (C) Andrew K. Milton 2007
# Released under a 2-clause BSD License
# See: http://www.opensource.org/licenses/bsd-license.php

import os
import time
import random

from optparse import OptionParser
from ConfigParser import SafeConfigParser
from ctypes import windll

from win32con import *
from PIL import Image

"""
I recommend you create a pywallpaper.conf file that looks something like this;
[directories]
paths = C:\Documents and Settings\akm\My Documents\My Pictures\gb
C:\Documents and Settings\akm\My Documents\My Pictures\Ralph

to store the directories in rather than specifying -d multiple times on the command line"""


def getDefaultDirs():
return [r'C:\Documents and Settings\All Users\Documents\My Pictures\Sample Pictures',]

def setWallPaperFromBmp(pathToBmp):
""" Given a path to a bmp, set it as the wallpaper """
result = windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER, 0,
pathToBmp,
SPIF_UPDATEINIFILE | SPIF_SENDWININICHANGE)
if not result:
raise Exception("Unable to set wallpaper.")


def setWallPaper(pathToImage):
""" Given a path to an image, convert it to bmp format and set it as the wallpaper"""

bmpImage = Image.open(pathToImage)
newPath = os.getcwd()
newPath = os.path.join(newPath, 'pywallpaper.bmp')
bmpImage.save(newPath, "BMP")
setWallPaperFromBmp(newPath)

def setWallPaperFromFileList(pathToDir):
""" Given a directory choose an image from it and set it as a wallpaper """
# Image directories often contain Thumbs.db or other non-image
# files or directories, try a few times to set a wallpaper, and then just give up.

tries = 0
done = False

while (not done) and tries < 3:
try:
files = os.listdir(pathToDir)
image = random.choice(files)
setWallPaper(os.path.join(pathToDir, image))
done = True
except:
tries += 1
return done

def setWallPaperFromDirList(pathList):
""" Given a list of directories choose a directory """
imageDir = random.choice(pathList)
status = setWallPaperFromFileList(imageDir)

# Don't really need to say anything if we couldn't set the
# wallpaper, I think the user will notice

def getImageDirectories(options):
configFile = 'pywallpaper.conf'

if options.configFile:
configFile = options.configFile

dirs = options.directories

try:
config = SafeConfigParser()
config.readfp(open(configFile))
dirs += config.get('directories', 'paths').split('\n')
except:
pass

if not dirs:
# raise Exception("No directories defined")
dirs = getDefaultDirs()
return dirs


def setWallPaperFromConfigDirs(options):
setWallPaperFromDirList(getImageDirectories(options))

def getCommandLineOptions():
parser = OptionParser()
parser.add_option("-t", "--time", dest="change_time",
help = "Change wallpaper time in minutes (0 = change once and exit [default])",
default = 0, type = "int")
parser.add_option("-i", "--image", dest="singleImage", default = None,
help = "Set wallpaper to this image and exit (overrides -d)")

parser.add_option("-d", "--directory", dest="directories", default = [],
action="append", type="string",
help = "Add an image directory")

parser.add_option("-c", "--config", dest="configFile", default = None,
help = "path to alternate config file (default <working dir>/pywallpaper.conf)")

parser.add_option("-w", "--workingdir", dest="cwd", default=".",
help = "Working Directory (default .)")

(options, args) = parser.parse_args()
return (options, args)

if __name__ == '__main__':

options, args = getCommandLineOptions()

if options.cwd and options.cwd != '.':
os.cwd(cwd)

if options.singleImage:
setWallPaper(options.singleImage)
elif not options.change_time:
setWallPaperFromConfigDirs(options)
else:
dirs = getImageDirectories(options)
sleepTime = options.change_time * 60.0
while True:
setWallPaperFromDirList(dirs)
time.sleep(sleepTime)

The more things change, the more they stay the same.

Things fall apart; the center cannot hold;

Mere anarchy is loosed upon the world,

The blood dimm'd tide is loosed, and everywhere

The ceremony of innocence is drowned;

The best lack all conviction, while the worst

Are full of passionate intensity.

-- W.B. Yeats

Zope and Trac

I've put together a Zope frontend for Trac.
It seems to work fine (at least for me).
You can find it here

Erin and Scott's Web Blog: Engagement ring shopping

Erin and Scott's Web Blog: Engagement ring shopping

Fads & Erin got engaged! OMG jas and I are so stoked for them.
We both think Fads' heart probably nearly failed when he asked.

Australian Eastern Timezones in Zope

Zope has had a well known bug with detecting Eastern Australian Timezones for a long long long time. Well today I got sick of detailing the workaround and created a Monkey Patch product for Zope 2 to fix it.

The problem is that Zope thinks that the EST and EDT timezones should be AEST and AEDT, well that's not what they're called, so most of the AU Zope servers run in US/Eastern time, which is very very very annoying.

So if you're in an Eastern AU state and you'd like your logs and times to be local without jumping through hoops, try downloading; ZAUTZ

More uploader stuff

Last night I had a lightbulb moment, and I added EPOZ support to the upload demo.
So you can use the wysiwyg editor, and add images from your HDD as you go.
I'll probably end up posting to the zope list for help with the IE javascript issues, since noone has responded on IRC or to the blog entry.

Async AJAX Uploader with progress bar for Zope

I've tidied up the Threaded Transfer Agent code somewhat, and added enough support into it to build an upload demo that shows asynchronous uploads in action. The screenshot shows multiple uploads in progress, from the one form, allowing content to continue being added. The status is updated from Zope via XMLRPC (using jsolait), so I guess it's technically an AJAX application.
Async Upload Demo in Action

I'll release this code soon, I've got to find some people to alpha test it, I'm sure I'll have a few volunteers.

Actually if you know a bit about JavaScript for IE, I'd be happy for you to contact me as some of the status updating code doesn't work so well for IE (runs like a charm in FireFox), so I can get an IE version of it working. The JS code itself does fairly simple things, but, doesn't work satisfactorily in IE.

No, it's not an April Fool's joke.

More side projects, and (small) Javascript grid rant.

I'm letting Compiled Page Templates settle for a little bit, I got some pretty quick feedback about some bugs, but, nothing for a little while, I'm sure there will be more. I'm hoping it gets a hammering by some people. I converted ZWiki to use it, but, the underlying framework is a little too obfuscated to really get a decent speed up out of it (I could only manage 10%).

Meanwhile I'm working on Javascript parser in python. I'm not sure where this will end up, but, I can see that it would be very useful to be able to run javascript inside python.

I've been trying to find a 'nice' looking template so I can actually build a 'proper' website. I've spent a few hours playing around building static pages, and I just can't be bothered. So I guess I'll being doing something in Zope for that too. Actually once I get that up and running, I think I might consolidate all of my various projects there. As always it's finding the time, and since we're being evicted, I don't think I'll make any headway on that until after we find a new place and move in.

I played a little with jsolait and Zope the other day. The jsolait xmlrpc lib is like python's so it's nice to use (even though I'm not a Javascript guy really), but, it's going to help out on my current paying gig, so it was worth investigating. I'm not really a front-end guy at all, but, it's irritating that even the 'nice' javascript grid widgets suck. I guess that's a byproduct of living inside a webpage. I really want to be able to create a grid that grows and shrinks dynamically. All of the ones I've seen so far, can grow, but, never get any smaller. That's great if you want to browse a fixed dataset, but, rotten if, for example, you want to display an active queue of items.

On the grid thing, the LiveGrid people suggested we no longer needed paging, and we could just use scrolling to navigate through datasets. Well that's great if your dataset is really small, but, it's sucks if you want to get to something in the middle of the dataset. Something that will stay in the middle of the dataset regardless of how you sort it. I often find myself binary searching results from various searches trying to find the set I want (I'm not talking google searches here). I don't want to go back to linear search.

Speeding up ZPTs

I'm doing a bit of 'tinkering' again with some of my old Zope projects I have lying around.
I''ve written my own TAL engine, and I was testing it to see if it was any faster;
Using this template (thanks mcdonc d8), I ran a test using the venerable ab tool that comes with apache. Don't write in to tell me my hardware is crap, I know that, I don't performance test on high-performance hardware..

<html xmlns:tal="http://xml.zope.org/namespaces/tal">
 <head>
   <title>This is the title</title>
   <div>This is the head slot</div>
 </head>
 <body tal:define="values python:[('bug', 'me'),]*20">
  <div>
    <span tal:replace="values" />
    <form action="." method="POST">
    <table border="0">
      <tbody>
        <tr tal:repeat="itemz values" class="foo">
        <td tal:content="python: itemz[0]">Name</td>
        <td tal:content="python: itemz[1]">Description</td>
        </tr>
      </tbody>
    </table>
    </form>
   </div>
 </body>
</html>

The following are directly to Zope, not via apache.
First standard Page Templates;

Server Software:        Zope/(Zope
Server Hostname:        xanadu
Server Port:            8080

Document Path:          /tmp/testTemplate2
Document Length:        2444 bytes

Concurrency Level:      10
Time taken for tests:   9.467635 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      2651000 bytes
HTML transferred:       2444000 bytes
Requests per second:    105.62 [#/sec] (mean)
Time per request:       94.676 [ms] (mean)
Time per request:       9.468 [ms] (mean, across all concurrent requests)
Transfer rate:          273.35 [Kbytes/sec] received

Connection Times (ms)
            min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       9
Processing:    20   93  13.1     93     189
Waiting:       18   87  12.5     88     169
Total:         20   93  13.2     93     189

Percentage of the requests served within a certain time (ms)
50%     93
66%     94
75%     94
80%     94
90%     99
95%    105
98%    143
99%    150
100%    189 (longest request)

And now my Templates.

Server Software:        Zope/(Zope
Server Hostname:        xanadu
Server Port:            8080

Document Path:          /tmp/testTemplate
Document Length:        2262 bytes

Concurrency Level:      10
Time taken for tests:   7.341181 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      2469000 bytes
HTML transferred:       2262000 bytes
Requests per second:    136.22 [#/sec] (mean)
Time per request:       73.412 [ms] (mean)
Time per request:       7.341 [ms] (mean, across all concurrent requests)
Transfer rate:          328.42 [Kbytes/sec] received

Connection Times (ms)
            min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       9
Processing:    23   72  13.6     72     170
Waiting:       21   66  12.7     67     130
Total:         23   72  13.6     72     170

Percentage of the requests served within a certain time (ms)
50%     72
66%     73
75%     73
80%     74
90%     87
95%     94
98%    107
99%    115
100%    170 (longest request)

30% faster, and that's a trivial page template. I need to find a less synthetic test, to do some 'real' testing and find out how much faster it really is, especially with METAL involved.
I can speed it up a little more too by sharing the 'cooking' between threads.

Anyhow stay tuned...