Funnelweb Comments to Disqus WXR

I've been meaning to migrate my blog comments out of Funnelweb and into Disqus for a while. I never quite got comments to work to my satisfaction in Funnelweb (user error I'm sure - Funnelweb is mostly great) and I appreciated the idea of subbing that out to a dedicated service. Plus I'll get the added benefit of having centrally located discussions that I can move to any blog system I decide to down the road with minimal fuss.

Disqus supports a few comment import formats (Wordpress, Blogger, etc.) and the generic WXR format which looked promising.

WXR is really just a specially formatted RSS XML file. The format that Disqus expects is described here. The simplest example I could find was here. It looks like this:


<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
 xmlns:content="http://purl.org/rss/1.0/modules/content/"
  xmlns:dsq="http://www.disqus.com/"
  xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:wp="http://wordpress.org/export/1.0/">
  <channel>
        <item>
          <title>Title of the article</title>
 <link>http://scottmetoyer.com/funnelweb-comments-to-disqus-wxr/</link>
          <!-- thread body; use cdata; html allowed (though will be formatted to DISQUS specs) -->
          <content:encoded>This is the article text.</content:encoded>
          <!-- value used within disqus_identifier; usually internal identifier of article -->
          <dsq:thread_identifier>237</dsq:thread_identifier>
          <!-- creation date of thread (article), in GMT -->
          <wp:post_date_gmt>2007-09-06 00:49:12</wp:post_date_gmt>
          <!-- open/closed values are acceptable -->

          <wp:comment_status>open</wp:comment_status>
          <wp:comment>

            <!-- internal id of comment -->
            <wp:comment_id>9001653</wp:comment_id>
            <!-- author display name -->
            <wp:comment_author>jips</wp:comment_author>
            <!-- author email address --> <wp:comment_author_email>no@email.com</wp:comment_author_email>
            <!-- author url, optional -->
            <wp:comment_author_url></wp:comment_author_url>
            <!-- author ip address -->
        <wp:comment_author_IP>127.0.0.1</wp:comment_author_IP>
            <!-- comment datetime, in GMT -->
            <wp:comment_date_gmt>2007-09-06 00:49:12</wp:comment_date_gmt>
            <!-- comment body; use cdata; html allowed (though will be formatted to DISQUS specs) -->
            <wp:comment_content>Comment goes here</wp:comment_content>
            <!-- is this comment approved? 0/1 -->
            <wp:comment_approved>1</wp:comment_approved>
            <!-- parent id (match up with wp:comment_id) -->
            <wp:comment_parent>0</wp:comment_parent>
          </wp:comment>
        </item>   
</channel>
</rss>

I put together a simple .NET console app to connect to a Funnelweb database, pull back all the comments, and output an XML comment file suitable for upload to Disqus. The project is on Github:

https://github.com/scottmetoyer/funnelweb-comments-to-wxr

To set it up, just rename My.config.sample to My.config and specify the connection string to your Funnelweb database and the root URL to your website (for creating article links). That's it - run the executable and you'll have a WXR file that can be uploaded to Disqus. I was able to move ~100 comments over with no issues.

Hopefully this helps make the switch a little easier for someone. Happy commenting.

RADIX16 Laptop Touchpad Matrix MIDI controller

radix16

Overview

The RADIX16 is a custom MIDI controller I built over the last year.

The controller is fashioned from 16 surplus laptop touchpads arranged in a 4x4 matrix. Each touchpad transmits:

  • Note on / note off when tapped
  • 0-127 MIDI CC for touch pressure
  • 0-127 MIDI CC for X axis position
  • 0-127 MIDI CC for Y axis position

For a total of 16 notes and 48 control change values available at the fingertips.

The number of parameters per pad allows for nuanced and expressive control over external synthesizers / effects. A pad can (for example) trigger a drum sample with variable velocity (z-index), filter cutoff (y-index), and pan (x-index) depending on where the pad surface is tapped. Here's a quick video of one of the first post-build tests - testing out the pads on the RADIX connected to a MicroKorg synthesizer:

The Build

I built the touchpad modules first, using surplus Synaptics touchpads sourced from PCHub. The touchpads came complete with ribbon cables - I attached the ribbon cables to small PCB adapters I designed and printed at Pad2Pad (see below for the CAD files and parts list). The output side of the adapter is a standard 8 pin header suitable for jumper wire.

Next came the front face panel for the controller. I considered a few designs and ultimately went with a three-piece "sandwich" of one backing panel of delrin (plastic), one middle panel of delrin to hold the touchpads, and one top panel of aluminum to lock everything together. The three panels were designed in Inkscape and sent over to Ponoko for laser cutting. The Ponoko template SVG files are below.

Here is the touchpad "sandwich" with the pads mounted and wired up - the aluminum top panel would be added last:

pads

The last piece of the build was the wood case. The case was constructed in my dad's workshop using scrap maple. We planed, cut, sanded, drilled, and stained it over the course of a weekend. I created and mounted an insert for the rear panel (power connector and switch, USB MIDI out) using a piece of angled aluminum.

case1

case2

The Electronics

The core of the electronics is an Arduino Mega 1280. An inexpensive USB MIDI interface is wired up to the Arduino TX pin to provide USB MIDI out. Conceptually this is what it looks like:

radix16 schematic

I created a custom shield that connects up all the touchpads, the MIDI interface, and external power to the Arduino.

Wiring a Synaptics touchpad to an Arduino is pretty simple. I used this Instructables article as a starting point. Each touchpad needs a dedicated clock and data pin which means 32 of the 54 available on the Mega are in use. I basically repeated that circuit 16 times.

This article describes wiring up a USB MIDI interface to an Arduino for easy MIDI out. I followed the same model.

I will get a schematic of the shield up soon.

pads 2

The Code

The software for this project was challenging.

The PS/2 protocol is well documented, but all of the sample code for doing this on an Arduino assumes only a single pad is connected. As such, the code is synchronous and will block while it waits for the touchpad clock. This means that the performance of the controller degrades proportionately with the number of pads that are connected - the more pads that are connected to the Arduino the greater the length of time between touching a pad and getting a MIDI message. With all 16 pads wired up latency was ~350 milliseconds - about 10 times too high. I had to get this down to a reasonable level.

Ultimately I ended up writing asynchronous PS/2 communication code from scratch. In addition I was able to do a few tricks and hacks (direct pin IO, unrolling loops) to get performance to an acceptable level. The final code is pretty ugly and will be harder to maintain, but latency is now around 50ms. The code is available for download below.

*** UPDATE *** Turns out I was a little pessimistic on my latency measurements. I set up some timers and and recorded an average latency of 32 ms - which is much better but not ideal. I have a few more optimizations to try and get this down further.

naked pads

finished

Files and Downloads

Arduino sketch

Ponoko delrin template SVG file

Ponoko aluminum faceplate SVG file

Adapter Pad2Pad PCB file

Cassette Scrubber

I decided to put to good use the a few of the hundreds of audio cassettes I've seen gathering dust in boxes around local thrift stores. This is the result:

scrubber

Construction was simple.

I picked up a lower than low-end cassette player from Amazon and tore out the tape head. The head is just a simple transducer and doesn't require any other circuitry. It has literally just two output wires (4 wires for stereo, my player was mono) that produce an unamplified audio signal. I wired the tape head outputs directly to a female 1/4" audio jack, plugged in a guitar cable, and ran it straight to the mic input on my mixer. Then I zip tied and screwed everything in to a 1' piece of scrap wood to form a tape head stylus.

scrubber 2

The next step was to create a tape palette to scrub the wand against. I cracked open a dozen thrift store cassette tapes, snipped out random 18" lengths of tape, and stretched ~100 of the strip across a piece of corkboard. I clamped more scrap wood across the edges to hold things tight. The result was about 200 square inches of tape containing random audio that I could tease out by sliding the tape head wand around.

scrubber 3

"Playing" the cassette scrubber requires some nuance. Despite the size of the head, the actual point that must make contact with the tape is quite small. If the head doesn't make direct contact with the tape, there's no sound produced. Also, it's close to impossible to move the head at a consistent speed (at least for me) so the sound warbles greatly. But that's part of the charm.

scrubber 4

I'll get audio samples and possibly a video up soon.

Contingency Reader

Contingency Reader is my host-it-yourself Google Reader replacement.

It was written over the course of about 10 days in C# / .NET 4.5 with SQL Server on the backend.

Contingency Reader

Features include:

  • Article retention
  • Starring / favorites
  • Atom 1.0 and RSS 2.0 feed support
  • Read history
  • Inifinite scroll (dynamic content load)

Download it from Github.

Feature suggestions are always welcome.

Quick and Dirty NZB indexing in Python

NZBMatrix is dead but I wasn't going to let that keep me from USENET.

While waiting for the dust to settle and a new indexing service to bubble up I wrote my own quick and dirty indexer in Python with MongoDB on the backend. Get it here.

The indexer takes a list of newsgroups to crawl as an input parameter, and searches for NZB files within those groups. Any article it finds that has '.nzb' in the title gets recorded in the database. The crawls are time stamped so only new articles are grabbed on each run.

I've got it set up to crawl 10 of my favorite groups every hour which usually takes under 5 minutes.

Next steps are to take the retrieved NZB article fragments and build up the actual NZB file. That will take some tinkering with yEnc... and may never happen now that I've found a decent replacement for NZBMatrix:

NZBGeek

Getting GitHub for Windows to play nice with PureData

PureData-extended was recently upgraded to 0.43.4. I couldn't wait to download and play around with some of the new features and check out the GUI updates.

After installation, my GitHub for Windows suddenly and inexplicably failed to clone, push, or pull remote repositories. It would only show this rather cryptic error message:

github error

Googling didn't turn anything up, and an email to Github for Windows helpdesk didn't provide a solution. They hadn't seen the issue before.

Some hints were discovered in the GitHub for Windows log, which can be viewed by pressing Windows Key - R (to bring up the Windows run dialog) and typing:


%LOCALAPPDATA%\GitHub\TheLog.txt

Doing a search for 'error' showed this message:


GitHub.IO.ProcessException: Cloning into 'daily-quote'...
error: Protocol https not supported or disabled in libcurl while accessing https://github.com/scottmetoyer/daily-quote.git/info/refs?service=git-upload-pack
fatal: HTTP request failed

There appeared to be something up with my libcurl library installation.

I remembered that the PureData installer had warned about overwriting some system files in the C:\Windows\System32 folder, so I went ahead and tried uninstalling PureData.

Surprisingly it worked! GitHub for Windows immediately started working again.

I still wanted to get my hands on the PureData upgrade though... fortunately, they have a non-installer (.zip) copy of the latest version on the download page. I downloaded, unzipped, and copied the files to a folder on my hard drive.

Success! Both PureData and GitHub for Windows ran together without error.

Now, off to make some music...