Ordo Ab Chao

Pulled this from DECAFBAD... Nice tasty article all about -X command line switch in mysql. Wonderful I thought. XML based power in the worlds most popular and free relational database. I heard the sounds of dreams coming true.. well maybe ;) There had to be a catch however.

However, just when I thought everything was peachy I started playing with this feature with less than spectacular results. Some intensive googling yielded the answer.
It seems that using the -X command line option for exporting the data in XML format produces invalid XML. It assumes XML escaped data in the DB!.. On what grounds??? Mysql only encloses the query results in XML element tags, but doesn't do encoding of the contents inside the tags.


In XML, if you want to use one of the characters <, >, &, etc. inside an element tag is not valid. If you want to use one of those characters, you have to use the respective entity instead. Mysql doesn't seem to do that, so when selecting tagged data or markup like "<foo>red & green</foo>" with the -X command line option will always lead to invalid XML.
An uncool workaround would be to perform some string replacements for every selected column when using the -X option:

• replace all & by &amp;


• replace all < by &lt;


• replace all > by &gt;


• replace all " by &quot;


• replace all ' by &#39;

In a previous life as a research manager in an Irish research group called TSSG I wrote a piece about semantic web for a technology column in a local paper. It's the usual non-critical high-level look at a technology but the excitement at the promise of semantic web is very real.


However I'm less than convinced about the current web services standardisation effort. In comments to another blog I was scathingly critical of the original WS technology (SOAP & XML-RPC) and the malaise of WS standards and specs. I've also been keenly following the wise words of Steve Vinoski, Chief Engineer of IONA technologies, another company I used to work for. Before I digress onto another topic entirely I'm going to reiterate some of my original comments about WS standardisation, mirroring steve's feelings about the lessons that can be learned from CORBA regarding tool & vendor support. So without wanting to offend too many of the great people involved in the process, here are my considered thoughts:

• "Web Services" is a brand name for a range of disparate and relatively unfocused technologies.
  


• The technology was hugely overhyped without accepted standards to back it up
  


• XML messages were touted as human-readable. If you know that many humans who read large XML schemas in their spare time you need to get yourself and your friends "to a nunnery". OK, maybe not but you get the point ;-)
  


• It often seems that around 20 years of distributed systems thinking was ignored in their creation. Hence SOAP was misnamed "Simple". "Incomplete" would have been more appropriate.
  


• With usefulness comes complexity. With complexity comes unwieldiness and with unwieldiness comes confusion. The secret is normally appropriate abstraction but it's early days yet
  


• The standardisation effort is frustrating and feels uncoordinated. All too often standards are hurriedly created to plug holes in other standards. Often if feels like the wheel is being reinvented, as if nobody in the effort knows that RPC has been done before. I hear Vinoski's cries for an overarching Architecture spec so have both a map and a flashlight
  


• Almost none of this matters as the major industry players are now behind it in a bid to recapture the goldrush of the late 90s with a 'must-have'technology. For this reasons alone the tool support will hide much of the complexity and encourage utilisation. This is already happening. Thank you Microsoft, IBM, HP, BEA, IONA, SUN etc.
  


• The most loosely coupled thing about WS/SOA is often the standardisation process. There could be trouble ahead
  

Yet another interesting nugget of information pulled from Jon Udell's site. Makes you wonder how many bloggers are merely human blog aggregators of other people's blogs. Eventually there's 1 part content and O(nⁿ) level of repetition, like P2P only worse as info is wrapped with 'opinion' by each subsequent blogger. There's a study that could be done here using a combination of the google API and bloglines. Blog information is distributed virally? Discuss...
E4X is native data type for XML in ECMAScript. More information here

The Internet was designed as a content access system which a predominantly client/server, assymetrically biased towards downstream (downloads etc.) With P2P exchange of data, the creation of decentralized groups allows for information to flow over the public Internet in an anonymous logical fashion. The individual users of these applications are shielded via this anonymity. There are obvious issues with IPR here but also more subtle issues regarding the categories and topology of P2P traffic. (I'll provide a more rigorous mathematical look on this soon) via this form of information exchange, the service providers no longer have the ability to forecast network capacity based on historical subscriber usage patterns. There are four key areas where service providers are feeling the pinch:

1. Upstream/downstream traffic is flipped where the upstream traffic is much larger then the downstream traffic. This results in network congestion on the upstream link that was never planned for with initial broadband deployments.


2. Time of day usage statistics no longer apply. Previously, service providers could assume peak usage at certain times of the day and lower usage at other times. With P2P applications, the computers are often left to transfer data throughout the day in an unattended fashion.


3. Previously, peering traffic always traversed the Internet to another location. In today’s world, two home users can form a direction connection.


4. Over-subscription assumptions no longer apply. A handful of power users can “hog” all of the bandwidth deployed for a much larger usage base.

This index was established in mid 90s and provides a statistical analysis of the degree of IT access and absorbtion within 53 countries worldwide. Ireland can only manage 23rd spot, which is less than impressive considering we're a small nation with such a disproportionate amount of our Gross Domestic Product (GDP) coming from IT. (For a cold hard look at our GDP/GNP comparisons read this) Our neighbours in the UK fare better in 10th, while the tech savvy danes and swedes claim 1st and 2nd place respectively.

Browser incompatibilities are definitely the bane of a web developer's life. Having spent much of my development life messing around with command lines, I'm now spending a lot of time looking ath CSS section of w3schools grabbling with CSS positioning & layout issues.


I decided that I'd solve some of these browser incompatibilites on the server side rather than with client side javascipt.. MT's natty Perl-plugin interface looked the best bet and I whipped up a few quick lines of PERL to pull the HTTP_USER_AGENT from the env and parse it. Easy-peasy I thought having read all about browser identities here (skipped the RFC)... This turned out to be no fun. I learned a lot about writing plugins which are a really great feature but when I outputted the browser ID for both IE and Opera I got guess what?
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Opera 7.54 [en]
Not exactly what I was expecting. A diff of the two confirmed that I wasn't going nuts. They're the same so my plugin is effectively useless for sorting out CSS layout issues between IExploder, Opera and Nutscrape... So HTTP_USER_AGENT is apparently not the thing to use.. The appName in javascript would be more reliable apparently. SO much for sorting out the problem on the server side. Ah well... de nouveau au conseil de dessin as they say in pidgin french :P

Following on from my earlier post about WS standardisation. Steve Vinoksi points out that traditional standardisation efforts are often too slow and overly political. In this month's IEEE Distributed Systems Online (DSO) he discusses WS-NonexistentStandards. Lots of standardisation work but where are the accepted standards and how does the process facilitate the creation and adoption of practical standards?

To get around these problems, WS-* authors appear to be taking a different approach toward standardization:

1. Write a specification and make it publicly available.


2. Invite interested parties to one or more private workshops where they can learn more details about the specification and provide feedback.


3. Iterate steps 1 and 2 until chosen feedback from the workshop participants has been incorporated, and the specification is considered finished.


4. Submit the specification to an official standards body with the hope of fast tracking it to actual standardization with minimal changes.

Overall, this approach reduces the number of participants involved, which can be a good thing because it reduces the overall volume of communication required to create the specification and resulting standard. However, it can also reduce the resulting standard’s effectiveness, even rendering it useless, because it circumvents at least some of the process of building consensus by not being a truly open process. A standard that is not generally agreed on is a standard on paper only.

1. The creation of WS-arch so we can categorically say what piece of the WS-jigsaw goes WS-where? ;-)


2. Incentivised involvement of independent s/w developers in the standardisation process. Spec consumers rather than spec producer/pushers who can't provide neutral guidance. Maybe even some decisions could be put to general developers using a web-based voting system.

Charlie Mills creates a Design-By-Contract library for C (which could equally be used for C++ with minor changes) in his most recent OnLamp article. DBC views functions and methods as contractual agreements between the functional caller and the object/module providing the function. Charlie's implementation is a really neat idea using Object Constraint Language (OCL)to describe:

• function preconditions


• function postconditions


• type and function invariants

Pulled this off benezedrine.cx. It a tasty, easy to replicate mechanism for dealing with spammers, safe in the knowledge that you're slowing down their grubby, stinking little operations in the process. As most spammers get paid by volume this reduces the money they make from slowing down the internet, helping to spread viruses and generally being complete assholes. The author advocates the creation of a tarpit using spamd which is basically an MTA which keeps SMTP relaying connections open but slows responses down to a C-R-A-W-L... Throw in the use of spamassasin for some dynamic spam detection together with the creation of a blacklist for tarpit redirection using information from an authoritative site like spews.org and you have a reliable system that kicks the majority of spammers where it hurts. The original text is shown below.
To quote Bill Hicks: "just trying to plant seeds"

Introduction
I don't like getting spam. The problem is not detecting it automatically, that works very well with tools like SpamAssassin and bmf . Even though I can automatically delete spam without reading it, the spammers still successfully deliver their mails and get paid by volume. I want to hurt them. They should not be able to deliver their mails, and waste as much of their resources as possible attempting to do so.

Tarpits
Tarpits like spamd are fake SMTP servers, which accept connections but don't deliver mail. Instead, they keep the connections open and reply very slowly. If the peer is patient enough to actually complete the SMTP dialogue (which will take ten minutes or more), the tarpit returns a 'temporary error' code (4xx), which indicates that the mail could not be delivered successfully and that the sender should keep the mail in his queue and retry again later. If he does, the same procedure repeats. Until, after several attempts, wasting both his queue space and socket handles for several days, he gives up. The resources I have to waste to do this are minimal.

If the sender is badly configured, an uncooperative recipient might actually delay his entire queue handling for several minutes each time he connects to the tarpit. And many spammers use badly configured open relays.

Obviously, I only want known spammers to get connected to my tarpit instead of my real MTA.

Blacklists
I can use an externally maintained list of spammers like spews.org to redirect senders to the tarpit selectively. But such lists may be either to slow to include new spamming hosts, or too aggressive for my taste. Some blacklists will not only include single hosts, but entire networks that contain a single spamming host, willingly hurting innocent customers of an ISP to pressure the ISP to terminate the spammer. The blacklist maintainers document such policies, and if I agree with them, it's my decision to block mail from such networks by using their blacklist.

But even if I'm comfortable with blocking mail from innocent bystanders and use the most aggressive blacklists combined, there will still be spammers getting mails delivered to me through newly discovered open relays. Those spam mails will of course be detected by my spam filters, so I'd like to use these IP addresses to build my own blacklist.

Building my own blacklist
Assume I have the following procmail configuration in place to detect (and file) spam:

:0fw
| /usr/local/bin/bmf -m maildir -p
:0:
* ^X-Spam-Status: Yes
in-x-spam
:0fw
| /usr/local/bin/spamc
:0:
* ^X-Spam-Status: Yes
in-x-spam

Each incoming mail is piped through the two spam detectors. If either one of them classifies the mail as spam, the message gets stored in a separate file. I could delete them instead, but I might want to check the mails for false positives every once in a while. Once the classifiers are tuned right, there will be almost no false positives, and almost all spam is detected. I'm reaching 99.95% accuracy here, with maybe 0.01% false positives, which is fine for me.

Analyzing Received: headers
I'm using one additional tool, relaydb , to build a database of all hosts that send me mail. This is done after the classification by the spam detectors, so I can tell the database whether the sender was sending spam or legitimate mail.

I add the following parts to my procmail configuration:

:0fw
| /usr/local/bin/bmf -m maildir -p
:0c
* ^X-Spam-Status: Yes
| /home/dhartmei/bin/relaydb -b
:0:
* ^X-Spam-Status: Yes
in-x-spam

:0fw
| /usr/local/bin/spamc
:0c
* ^X-Spam-Status: Yes
| /home/dhartmei/bin/relaydb -b
:0:
* ^X-Spam-Status: Yes
in-x-spam

:0c
| /home/dhartmei/bin/relaydb -w

So, detected spam gets piped through relaydb -b (blacklist), and legitimate mail through relaydb -w (whitelist). Note that only copies of mails get piped through relaydb, the program never modifies or drops a mail. All it does is build a database of hosts that sent me mail, counting spam and legitimate mail from each one.

relaydb traverses all Received: headers in a mail from top (nearest relay) to bottom. It only acts on valid numerical IP addresses in [] brackets, which is the only reliable part. And it's only reliable when I trust the previous relay in the chain, as spammers often add fake Received: headers. So relaydb starts with the top-most relay in the header and consults its database to see whether it is a known host, and if so, whether it sent me legitimate mail before. If that's the case, it increases the respective counter (spam or legitimate, as told through the -b/-w option) for that host and continues with the next relay found in the header. If the relay is a known spammer, traversal ends, as further headers cannot be trusted.

After I run this setup for a while, relaydb has built both a blacklist and a whitelist. One important detail is that a legitimate mail has more weight than than a spam mail. I regularly receive spam through mailing lists. Of course, I don't consider the mailing list server a spamming host. Yet, each spam I receive through it will increase the spam counter for that server. Therefore, relaydb only reports hosts as blacklisted when their spam counter is at least three times as high as the counter for legitimate mail (and the factor can be adjusted, of course). So a relay doesn't get blacklisted as long as it sends me legitimate mail to compensate for spam it sends, which covers mailing list servers. But if I get a spam from a host that never sent me anything before, that will cause it to get blacklisted immediately (1 >= 0*3).

Completing the puzzle
Now I'm building my own blacklist, based on the evidence I've seen myself, classified by my own spam detector configuration. The only politics involved in someone getting blacklisted are my own, I don't have to trust a third party to make fair decisions.

And I use this blacklist to redirect hosts to the tarpit, using pf and some cronjobs:


$ pfctl -sn
rdr inet proto tcp from to any port 25 -> 127.0.0.1 port 8025

$ relaydb -lb | pfctl -t spammers -T replace -f -

This requires a recent OpenBSD -current system.

Instead of just loading the relaydb blacklist to redirect to spamd, I could combine it with spews. Or I can use the whitelist to prevent hosts which have sent me legitimate mail before from getting redirected to spamd due to a spews listing, etc. There are many interesting combinations.

And how well does it work?
I'm getting several dozen connections redirected to the tarpit per hour, and most peers waste about ten minutes per connection, and retry several times, for multiple days. The impact on my own resources is minimal.

Best of all, I regularly get spam through a mailing list and the sender (not the mailing list server!) gets blacklisted. Then the same spammer connects to me directly, too, as it harvested my address like the one of the mailing list. And it gets stuck in the tarpit. For long. And many times.

Remember, I'm doing all of this not to reduce the amount of incoming spam. That gets detected and filed very reliably, anyway. The sole purpose is to hurt the spammers. And I'm thoroughly enjoying watching my spamd log now, as I'm perfectly sure that each of those connections comes from a spammer who has spammed me before.

"Spam me once, shame on you. Spam me twice, shame on me." :)

If you have questions or comments, write to daniel@benzedrine.cx . And all you spammers harvesting email addresses from pages like this, please spam me. My trap is awaiting you.

The Federal Communications Commission (FCC) have decided that individual states cannot impose additional restrictions on VoIP service providers. This follows an attempt by the Minneapolis public utilities commission to force Vonage to abide by the same rules as existing telephony service providers. The FCC overruled deeming that this stance was "inconsistent with the FCC's deregulatory policies". More information on the reg. This is a fascinating story as the implications of this ruling are unclear. The FCC's policies to-date regarding VoIP are supportive but not coherent. It's very much a wait-and-see approach rather than a strategy promoting adoption of VoIP while reasonably compensating existing operators for the user of their network. This kind of sustainable policy is required to ensure that VoIP services are deployed in a safe and responsible manner with the reliability and security that users expect.

This post is actually about U2's new record "How to Dismantle an Atomic Bomb" which is currently spreading like wildfire on certain well-known P2P networks. The problem, apart from the obvious copyright infringements, is that the record hasn't even been released yet. It's due for release the 22^nd of November. However, a copy of the album dissappeared at a photo shoot and since then there's been intense speculation about whether the band would bring forward the release date. No decision has been made as yet. More info at the reg..

I've been told that I should add more of the little programming hints and tips that I used to come up with during my reearch days to this site. Well here's something I was playing around with today that's useful for many windows developers. Like many programmers I'm often more comfortable at the command line than using some funky GUI where I have to drag (or learn so many command alias key-strokes that I may aswell be at the console anyway).
I was stuck for a UNIX version of the which command. According to man which this command

Which takes a series of program names, and prints out the
full pathname of the program that the shell would call to
execute it. It does this by simulating the shells search-
ing of the $PATH environment variable.

Replicating this functionality using DOS batch ain't that bad...

@ECHO OFF
rem Sanity check OS version and arguments.
IF "%OS%"=="Windows_NT" (SETLOCAL) ELSE (GOTO Syntax)
IF "%~1"=="" GOTO Syntax
IF NOT "%~2"=="" GOTO Syntax
ECHO.%1 ¦ FIND /V ":" ¦ FIND /V "\" ¦ FIND /V "*" ¦ FIND /V "?" ¦ FIND /V "," ¦ FIND /V ";" ¦ FIND /V "/" ¦ FIND "%1" >NUL
IF ERRORLEVEL 1 GOTO Syntax

SET Found=
rem Get the short name for the current directory
COMMAND /C REM
rem Search CurrentDir, path and pathext for the file
FOR %%A IN (%CD%;%Path%) DO FOR %%B IN (.;%PathExt%) DO IF EXIST "%%~A.\%~1%%~B" CALL :Found "%%~A.\%~1%%~B"
rem Display the result
ECHO.
IF DEFINED Found (ECHO.%Found%) ELSE (ECHO -None-)
rem Done
GOTO End

:Found
IF DEFINED Found GOTO:EOF
rem Store the first match found
SET Found=%~f1
GOTO:EOF
:Syntax
ECHO.
ECHO WHICH, Version 2.00
ECHO UNIX-like WHICH utility for Windows NT 4 / 2000 / XP
ECHO.
ECHO Usage: WHICH program_name
ECHO.
ECHO Specify program_name with or without
ECHO extension and without a drive or path.
ECHO Just like the UNIX command. (no wildcards please)

:End
IF "%OS%"=="Windows_NT" ENDLOCAL


Martin Fowler has a neat little article on refactoring class statics using instance variables. Most languages can't support polymorphism for static methods. e.g.

class A{
public void doInitStuff() { /*do stuff necessary for static init of B objects*/};
} ...
}
class B extends A{
public void doInitStuff() { /*do other stuff necessary for static init of B objects*/};
} ...
}
...
A a = new B();
A.doInitStuff(); /* but I'd quite like to polymorphically call B.doStuff(); Actually, could be trouble! */


Martin's solution is very elegant.

Anybody who's done a bit of device driver development will know that occasionally system logs just don't provide enough information about the various problems you'll encounter and you have to hack up a shim library which sits between a problem library and it's loader/calling module. Linux Journal has a very nice article this month on creating just such a library for libusb. This could be useful for anybody developing an application or driver which needs to communicate with a USB device. Like writing a synch for a PDA, MP3 player or somesuch..

Japanese Ratoc Systems Corp. has a new lineup of wireless audio products called the "REX-Link" series, and some of them are even specifically designed to fit your 3/4G iPod or iPod mini.

There are four products in the series—two "receivers" and two "transmitters." Receivers come in the form of the "CR-RX01" with optical and analog audio outputs or the "REX-WHP1" headphones. For your transmitter, you have two options: the "CR-TXB01" USB transmitter, or "CR-TXB02" USB/analog transmitter (which also attaches to the back of 3/4G/mini iPods). These four products are matched to give three available packages: one with CR-TXB02 analog/USB transmitter and CR-RX01 receiver, one with CR-TBX01 USB transmitter and REX-WHP1 headphones, and one with CR-TXB02 analog/USB transmitter and REX-WHP1 headphones.

Read about Microsoft Shell a few months ago and it seems like it's getting some serious attention. (maybe not as much as MS's search engine pitch but I'll hold fire for the moment).
MSH is a genuinely great idea from Microsoft. Not an unusual thing in itself but this is a bit different. To quote Udell

System administration has always been Windows' Achilles' heel. The graphical tools that simplify basic chores just get in the way when there's heavy lifting to be done. And CMD.EXE, the hapless command shell, pales in comparison to the Unix shells that inspired it. Win32 Perl has been my ace in the hole, combining a powerful scripting language with extensions that can wield Windows' directory, registry, event log, and COM services. But I've always thought there should be a better way.

Jeffrey Snover thought so, too. He's the architect of Monad, aka MSH (Microsoft Shell), the radical new Windows command shell first shown at the Professional Developers Conference last fall.

MSH is quirky, complex, delightful, and utterly addictive. You can, for example, convert objects to and from XML so that programs that don't natively speak .Net can have a crack at them. There's SQL-like sorting and grouping. You write ad hoc extensions in a built-in scripting language that feels vaguely Perlish. (sd: reminds me a bit of bash scripting) For more permanent extensions, called cmdlets, you use .Net languages.

Just reading the complete Udell article again.
Can't help but feel that getting an XML representation of system processes over a certain size using a command like: <

MSH> get-process | pick-object name,vs | where { $_.vs -gt 150000000} | convert-xml

is extremely neat. Sample results are listed below. I'm less than convinced about the two-part name/type syntax for the XML representation (it's a bit clunky) but this is a small quibble.


<MshObjects>
<MshObject ReferenceID="ReferenceId-0" Version="1.1">
<MemberSet>
<Note Name="name" IsHidden="false" IsInstance="true" IsSettable="true">
<string> firefox</string>
</Note>
<Note Name="vs" IsHidden="false" IsInstance="true" IsSettable="true">
<int> 220983296</int>
</Note>
</MemberSet>
</MshObject>
</MshObjects>


While writing the last post I didn't fancy the idea of hand escaping the HTML entities into the MSN XML output. So I cheated using a funky little piece of XSLT that I cooked up earlier tonight. It's listed below...

<xsl:stylesheet version="1.0">
<xsl:output method="xml" indent="yes" omit-xml-declaration="no" doctype-
system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" doctype-public=
"-//W3C//DTD XHTML 1.0 Transitional//EN"> </xsl:output>
<xsl:template match="/">
<xsl:call-template name="escapexml">
<xsl:with-param name="block" select="."> </xsl:with-param>
</xsl:call-template>
</xsl:template>
<xsl:template name="escapexml">
<xsl:param name="block"> </xsl:param>
<xsl:for-each select="$block/*|$block/text()">
<xsl:choose>
<xsl:when test="self::text()">
<xsl:value-of select="."> </xsl:value-of>
</xsl:when>
<xsl:otherwise>
<xsl:text> &lt;</xsl:text>
<xsl:value-of select="name(.)"> </xsl:value-of>
<xsl:for-each select="@*">
<xsl:value-of select="concat(' ', name())">
</xsl:value-of>
<xsl:text> ="</xsl:text>
<xsl:value-of select="."> </xsl:value-of>
<xsl:text> "</xsl:text>
</xsl:for-each>
<xsl:text> &gt; </xsl:text>
<xsl:choose>
<xsl:when test="*">
<xsl:call-template name="escapexml">
<xsl:with-param name="block" select=".">
; </xsl:with-param>
</xsl:call-template>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="."> </xsl:value-of>
</xsl:otherwise>
</xsl:choose>
<xsl:text> &lt;/</xsl:text>
<xsl:value-of select="name(.)"> </xsl:value-of>
<xsl:text> &gt; </xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>

I'll produce a tidier more beautifying version when I get the time but it's OK for a first attempt and I know of at least one person who's asked me to do something similiar in the past. And on that note.. it could be time to hit the hay ;-)

Picked up a useful piece of information from Xyling Java blog about Java 6 becoming increasingly open source
Building on the success of the snapshot release process started during the development of J2SE 5.0, SUN will make early snapshots available for J2SE 6. This will afford developers the chance to make a greater contribution to the Java development lifecycle. Mb>Only fair considering developers generally have an idea or two about improvements they'd like to see in languages, libraries, API's etc. Well done to SUN on this developer friendly initiative.
(thinking of coming up with a "developer-friendly" logo similiar to the dolphin friendly one you see on some cans of tuna! Put suggestions on the comments page )

Save time writing a difficult and boring java style guide! Refer to William Blake's handy page which covers everything from Java exceptions to method and variable naming. I used to have something similiar in my lecturing days but this version is definitely better.

Just read an interesting post on Steven Vinoski's middleware matters about the lack of multiple-port support in the End-Point Reference EPR currently under review by the WS-Addresing working group and augmenting WSDL 1.1 by allowing for more dynamic usage patterns. Currently the EPR doesn't suport multiple ports. Ports, for those that remember the original WSDL spec, enable a webservice to be accesible through multiple protocol/transport/format alternatives. Steve Vinoski proposes a useful "business card" analogy to explain the practicality of multiple ports, covered by one EPR. Personal addressing on the internet has arguably evolved this way anyway. Here are some examples of ports associated with the person Shane Dempsey (of Geesan Tech, for SPAM avoidance purposes) with basic URI: sdempsey@geesan.com

• EMAIL: mailto:sdempsey@geesan.com (TCP, port 25)


• MULTIMEDIA SESSIONS: sip:sdempsey@geesan.com (UDP/TCP, port 5060)


• SIP INSTANT MESSAGING: im:sdempsey@geesan.com (UDP, port 5060)


• JABBER INSTANT MESSAGING: sdempsey@geesan.com (TCP, port 5222)

Picked up the following link from Jon Udell about the CAPTCHA (Computer Automated Public Turing test to tell Computers & Humans apart) preventing blog spam. This is a really (should that be raelly) tasty idea from Rael Dornfest. It can be summed up as can't add, can't post. He uses the Blosxom Writeback function which provides weblog comments with write-backs. An arithmetic sum is embedded in the writeback and no commets are allowed unless the answer is posted correctly. An example of this is
5 + 2 =
Neatly sidestepping more general blog spambots. The numbers are generated randomly. A definite improvement would be image obfuscation (a la Captcha!) and a bigger range.. He currently only uses 0-9 meaning a 1 in 20 chance you're gonna get the right number. I'm not sure I want to encourage blogspammers to brute force my site, especially when a post is so tantalisingly close
I'm working on my own interesting weapon in the battle against blogspam. It currently has the catchy title of blogassasin (Apologies to jmason & the rest of the spamassasin team). Also, it doesn't kill blogs but early versions come close. Active blacklist generation is another tidy feature. So spammers should think before thy HTTPiss Off innocent bloggers. Personally I don't believe that my blog (or anyone else's for that matter) needs to become any less relevant or increasingly grbled. So let's say NO to blogspam ;-)

It had to happen at some stage I suppose. (although I'm still not quite sure why???). Blogshares enables users to trade blogshares similiar to a fantasy stockbroker game.
Blogs are assigned monetary values based on the number of incoming and outgoing links to other blogs. It's similiar to Google in that it measures 'connectedness' Currently this blog is very lowly ranked :( Probably due to the fact that most of my friends don't actually maintain blogs so the usual web of trackbacks is avoided. It could also be that I haven't said anything interesting. I hope not. Also my blogroll is generated using javascript and it appears that the blogshare parser has failed to pick up these links...

Evolutiontwo has a profane and funny response to all spammers. I hear you brother. Like the rest of the sane world he has no intention of passing his bank a/c details over to some spammer claiming to be from Africa, buying a fake rolex or using a super-cheap online pharmacy to buy drugs to enlarge various body parts. Also, to give a lot of net user's credit they're bright enough to know that it was a spammer rather than an online lottery that harvested their email address.

I was playing around with regular expressions in Java. AFAIK these are only around since the JDK 1.4 and are therefore quite new. As a sometimes Perl programmer I've some experience with these but .
Hoever, all this hacking reminded me of the most amazing regular expression I ever saw. I saw this on the ActiveState's RX cookbook some time ago.
It's actually a useful and logically sound solution to a common problem... How to match all RFC 1738 compliant URLs and turn them into hyperlinks! It was posted by Abigail to comp.lang.perl.misc on 08/14/2000. Abigail, I love you!!!


$string =~ s<
(?:http://(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.
)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)
){3}))(?::(?:\d+))?)(?:/(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F
\d]{2}))|[;:@&=])*)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{
2}))|[;:@&=])*))*)(?:\?(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{
2}))|[;:@&=])*))?)?)|(?:ftp://(?:(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?
:%[a-fA-F\d]{2}))|[;?&=])*)(?::(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-
fA-F\d]{2}))|[;?&=])*))?@)?(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-
)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?
:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?))(?:/(?:(?:(?:(?:[a-zA-Z\d$\-_.+!
*'(),]|(?:%[a-fA-F\d]{2}))|[?:@&=])*)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'()
,]|(?:%[a-fA-F\d]{2}))|[?:@&=])*))*)(?:;type=[AIDaid])?)?)|(?:news:(?:
(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[;/?:&=])+@(?:(?:(
?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[
a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3})))|(?:[a-zA-Z](
?:[a-zA-Z\d]|[_.+-])*)|\*))|(?:nntp://(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[
a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d
])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?)/(?:[a-zA-Z](?:[a-zA-Z
\d]|[_.+-])*)(?:/(?:\d+))?)|(?:telnet://(?:(?:(?:(?:(?:[a-zA-Z\d$\-_.+
!*'(),]|(?:%[a-fA-F\d]{2}))|[;?&=])*)(?::(?:(?:(?:[a-zA-Z\d$\-_.+!*'()
,]|(?:%[a-fA-F\d]{2}))|[;?&=])*))?@)?(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a
-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d]
)?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?))/?)|(?:gopher://(?:(?:
(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:
(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+
))?)(?:/(?:[a-zA-Z\d$\-_.+!*'(),;/?:@&=]|(?:%[a-fA-F\d]{2}))(?:(?:(?:[
a-zA-Z\d$\-_.+!*'(),;/?:@&=]|(?:%[a-fA-F\d]{2}))*)(?:%09(?:(?:(?:[a-zA
-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[;:@&=])*)(?:%09(?:(?:[a-zA-Z\d$
\-_.+!*'(),;/?:@&=]|(?:%[a-fA-F\d]{2}))*))?)?)?)?)|(?:wais://(?:(?:(?:
(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:
[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?
)/(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))*)(?:(?:/(?:(?:[a-zA
-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))*)/(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(
?:%[a-fA-F\d]{2}))*))|\?(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]
{2}))|[;:@&=])*))?)|(?:mailto:(?:(?:[a-zA-Z\d$\-_.+!*'(),;/?:@&=]|(?:%
[a-fA-F\d]{2}))+))|(?:file://(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]
|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:
(?:\d+)(?:\.(?:\d+)){3}))|localhost)?/(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'()
,]|(?:%[a-fA-F\d]{2}))|[?:@&=])*)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(
?:%[a-fA-F\d]{2}))|[?:@&=])*))*))|(?:prospero://(?:(?:(?:(?:(?:[a-zA-Z
\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)
*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?)/(?:(?:(?:(?
:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[?:@&=])*)(?:/(?:(?:(?:[a-
zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[?:@&=])*))*)(?:(?:;(?:(?:(?:[
a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[?:@&])*)=(?:(?:(?:[a-zA-Z\d
$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[?:@&])*)))*)|(?:ldap://(?:(?:(?:(?:
(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?:
[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))?
))?/(?:(?:(?:(?:(?:(?:(?:[a-zA-Z\d]|%(?:3\d|[46][a-fA-F\d]|[57][Aa\d])
)|(?:%20))+|(?:OID|oid)\.(?:(?:\d+)(?:\.(?:\d+))*))(?:(?:%0[Aa])?(?:%2
0)*)=(?:(?:%0[Aa])?(?:%20)*))?(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F
\d]{2}))*))(?:(?:(?:%0[Aa])?(?:%20)*)\+(?:(?:%0[Aa])?(?:%20)*)(?:(?:(?
:(?:(?:[a-zA-Z\d]|%(?:3\d|[46][a-fA-F\d]|[57][Aa\d]))|(?:%20))+|(?:OID
|oid)\.(?:(?:\d+)(?:\.(?:\d+))*))(?:(?:%0[Aa])?(?:%20)*)=(?:(?:%0[Aa])
?(?:%20)*))?(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))*)))*)(?:(
?:(?:(?:%0[Aa])?(?:%20)*)(?:[;,])(?:(?:%0[Aa])?(?:%20)*))(?:(?:(?:(?:(
?:(?:[a-zA-Z\d]|%(?:3\d|[46][a-fA-F\d]|[57][Aa\d]))|(?:%20))+|(?:OID|o
id)\.(?:(?:\d+)(?:\.(?:\d+))*))(?:(?:%0[Aa])?(?:%20)*)=(?:(?:%0[Aa])?(
?:%20)*))?(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))*))(?:(?:(?:
%0[Aa])?(?:%20)*)\+(?:(?:%0[Aa])?(?:%20)*)(?:(?:(?:(?:(?:[a-zA-Z\d]|%(
?:3\d|[46][a-fA-F\d]|[57][Aa\d]))|(?:%20))+|(?:OID|oid)\.(?:(?:\d+)(?:
\.(?:\d+))*))(?:(?:%0[Aa])?(?:%20)*)=(?:(?:%0[Aa])?(?:%20)*))?(?:(?:[a
-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))*)))*))*(?:(?:(?:%0[Aa])?(?:%2
0)*)(?:[;,])(?:(?:%0[Aa])?(?:%20)*))?)(?:\?(?:(?:(?:(?:[a-zA-Z\d$\-_.+
!*'(),]|(?:%[a-fA-F\d]{2}))+)(?:,(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-f
A-F\d]{2}))+))*)?)(?:\?(?:base|one|sub)(?:\?(?:((?:[a-zA-Z\d$\-_.+!*'(
),;/?:@&=]|(?:%[a-fA-F\d]{2}))+)))?)?)?)|(?:(?:z39\.50[rs])://(?:(?:(?
:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?)\.)*(?:[a-zA-Z](?:(?
:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:\d+)){3}))(?::(?:\d+))
?)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))+)(?:\+(?:(?:
[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))+))*(?:\?(?:(?:[a-zA-Z\d$\-_
.+!*'(),]|(?:%[a-fA-F\d]{2}))+))?)?(?:;esn=(?:(?:[a-zA-Z\d$\-_.+!*'(),
]|(?:%[a-fA-F\d]{2}))+))?(?:;rs=(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA
-F\d]{2}))+)(?:\+(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))+))*)
?))|(?:cid:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[;?:@&=
])*))|(?:mid:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[;?:@
&=])*)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[;?:@&=]
)*))?)|(?:vemmi://(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z
\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:.(?:\d+)){3}))(?::(?:\d+))?)(?:/(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a
-fA-F\d]{2}))|[/?:@&=])*)(?:(?:;(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a
-fA-F\d]{2}))|[/?:@&])*)=(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d
]{2}))|[/?:@&])*))*))?)|(?:imap://(?:(?:(?:(?:(?:(?:(?:[a-zA-Z\d$\-_.+
!*'(),]|(?:%[a-fA-F\d]{2}))|[&=~])+)(?:(?:;[Aa][Uu][Tt][Hh]=(?:\*|(?:(
?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[&=~])+))))?)|(?:(?:;[
Aa][Uu][Tt][Hh]=(?:\*|(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2
}))|[&=~])+)))(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[
&=~])+))?))@)?(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])
?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:\.(?:
\d+)){3}))(?::(?:\d+))?))/(?:(?:(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:
%[a-fA-F\d]{2}))|[&=~:@/])+)?;[Tt][Yy][Pp][Ee]=(?:[Ll](?:[Ii][Ss][Tt]|
[Ss][Uu][Bb])))|(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))
|[&=~:@/])+)(?:\?(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[
&=~:@/])+))?(?:(?:;[Uu][Ii][Dd][Vv][Aa][Ll][Ii][Dd][Ii][Tt][Yy]=(?:[1-
9]\d*)))?)|(?:(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[&=~
:@/])+)(?:(?:;[Uu][Ii][Dd][Vv][Aa][Ll][Ii][Dd][Ii][Tt][Yy]=(?:[1-9]\d*
)))?(?:/;[Uu][Ii][Dd]=(?:[1-9]\d*))(?:(?:/;[Ss][Ee][Cc][Tt][Ii][Oo][Nn
]=(?:(?:(?:[a-zA-Z\d$\-_.+!*'(),]|(?:%[a-fA-F\d]{2}))|[&=~:@/])+)))?))
)?)|(?:nfs:(?:(?://(?:(?:(?:(?:(?:[a-zA-Z\d](?:(?:[a-zA-Z\d]|-)*[a-zA-
Z\d])?)\.)*(?:[a-zA-Z](?:(?:[a-zA-Z\d]|-)*[a-zA-Z\d])?))|(?:(?:\d+)(?:
\.(?:\d+)){3}))(?::(?:\d+))?)(?:(?:/(?:(?:(?:(?:(?:[a-zA-Z\d\$\-_.!~*'
(),])|(?:%[a-fA-F\d]{2})|[:@&=+])*)(?:/(?:(?:(?:[a-zA-Z\d\$\-_.!~*'(),
])|(?:%[a-fA-F\d]{2})|[:@&=+])*))*)?)))?)|(?:/(?:(?:(?:(?:(?:[a-zA-Z\d
\$\-_.!~*'(),])|(?:%[a-fA-F\d]{2})|[:@&=+])*)(?:/(?:(?:(?:[a-zA-Z\d\$-_.!~*'(),])|(?:%[a-fA-F\d]{2})|[:@&=+])*))*)?))|(?:(?:(?:(?:(?:[a-zA-
Z\d\$\-_.!~*'(),])|(?:%[a-fA-F\d]{2})|[:@&=+])*)(?:/(?:(?:(?:[a-zA-Z\d
\$\-_.!~*'(),])|(?:%[a-fA-F\d]{2})|[:@&=+])*))*)?)))
><a href = "$1">$1</a>>gx;

Needless to say, anything this complex requires a license to say that it may not work which is reprinted here (even though it logically should work at all times). Wow and wholeheated respect to Abigail...

If you want to match URL's reliably without creating a regexp monster so big that you need to connect up the digital projector just so you can work on it then this is something tasty I've come up with. Demonstrated in java, my language of choice


Pattern urlPattern = Pattern.compile("(((URL:|url:|http:|htt:)\\/\\/)|www\\.)(((([A-Za-z0-9][A-Za-z0-9-]*[A-Za-z0-9]"+
"|[A-Za-z0-9])\\.)*([a-zA-Z][A-Za-z0-9-]*[A-Za-z0-9]|[a-zA-Z]))|([0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+))"+
"(:[0-9]+)?(\\/([a-zA-Z0-9$_.+!*'(,);:@&=\\~\\#-]|%[0-9A-Fa-f][0-9A-Fa-f])*(\\/([a-zA-Z0-9$_.+!*'(,)"+
";:@&=\\~\\#-]|%[0-9A-Fa-f][0-9A-Fa-f])*)*(\\?([a-zA-Z0-9$_.+!*'(,);:@&=\\~\\#-]|%[0-9A-Fa-f][0-9A-Fa-f])*)?)?)";
Matcher urlMatcher = urlPattern.matcher("http://streamserver.gaisan.com/ourapplication?sd=234324&cam=1");
boolean matches2 = m2.matches();
System.out.println("Match should be true:\t" + urlMatcher.matches());


Perhaps not. I've commented in the past about the effects that P2P networks have upon the ISP traffic topologies (timing, upstream/downstream biases etc.) and we all know they can be used to illegally share copyrighted files. However, I strongly believe that P2P applications are the prototype for the next generation of highly resilient and scalable internet applications. In my former job as a telecomms researcher at TSSG we came up with quite a novel approach to integrating active networking and peer-2-peer apps at the top of the stack. I'm not sure what became of that work but my faith in the technology hasn't waivered.
I guess that's why I was so fascinated by the following post on boing-boing about 2 Princeton researchers who've cooked up a P2P app in 15 lines of concise Python code. The original post is located on Ed Felton's blog. It was damn funny to see someone hack up a Perl version in 9 lines. Without disrespect, the python implementation is more legible but the Perl code wins my "tight code" Award for 2004. Matthew Scala has a well used styrofoam cup with an strategicaly embedded 1/2 fried 2Mb Dimm (circa 1993) winging its way to him at this very moment. Enjoy! What a prize and what a hack :D

While I was typing the last entry I wondered if there were any websites devoted to old PC technology. My sense of nostalgia overwhelmed me when I visited Old-Computers.com. In particular this article on the IBM PC AT brought it all back. I remember using one of these in school when I was younger. It had the 286 processor (which really kicked ass in its day) an outlandish 1Mb of RAM and 16-bit expansion slots, of course. This had a type 2 mobo with 4 standardised 256k slots instead of 128.. Those were the days.. when computers were dumb, real men used DOS, we played footie in the park, jumpers for goalpsots...
Apparently the 128 k slots were a bit weirder than they seemed initially.

The first AT used 128 k chips, which appeared to be two 64k chips stacked. It used two DMA chips, which tended to fail in tandem. It also used a second IRQ controller. If the AT had more than 640 k of RAM, the CMOS would only allocate the first 512 as Convential, the rest as Extended.
Only 17 hard drive type were supported in the CMOS, causing no end of headaches when Seagate realsed their 40 meg half height. The 1.2 meg floppy drive could read and write 360's, but if you formatted one, it couldn't be read by a regular double density drive.

I must confess to being absolutely fascinated by headmap.org. I'm fascinated by the idea of smart spaces which infer user intent based on learned context. For example an office space that learns that automatically adjusts the heating in a room based on predictions about a meeting occuring. Lights that switch themselves off when there's nobody around etc. The most value is achieved when the ambient intelligence is fully integrated with other organisational information systems such as email and IM servers, project management tools, data and profile repositories etc. I fancy the idea that every node in an increasingly networked world could dynamically negotiate new cooperative strategies and operations based on an understanding of user intent. A true User Oriented Architecture. This could be communicated using a standardised information markup with transforms for hetergeneous devices to address capability differences. I'm straying into agents territory here but there was a lot of value in that research. In essence,extending the human computer interface (HCI) throughout the user environment. In particular I like the idea about capturing memories at locations, augmenting the real world with location/memory tags, a bit like the virtual worlds created by multi-player games. The possibilities are amazing & the results would perhaps be indistinguishable from magic...

This is getting ridiculous. Following my recent post about Ed Shelton's P2P program in 15 lines of Perl there's been a P2P app done in 9 lines of Perl and now (wait for it) a full peer-2-peer application in 6 lines of Ruby with a 3 lines of comments.
Just to show everybody how nuts this has become the code is reproduced below...

# Server: ruby p2p.rb password server server-uri merge-servers
# Sample: ruby p2p.rb foobar server druby://localhost:1337 druby://foo.bar:1337
# Client: ruby p2p.rb password client server-uri download-pattern
# Sample: ruby p2p.rb foobar client druby://localhost:1337 *.rb
require'drb';F,D,C,P,M,U,*O=File,Class,Dir,*ARGV;def s(p)F.split(p[/[^|].*/])[-1
]end;def c(u);DRbObject.new((),u)end;def x(u)[P,u].hash;end;M=="client"&&c(U).f(
x(U)).each{|n|p,c=x(n),c(n);(c.f(p,O[0],0).map{|f|s f}-D["*"]).each{|f|F.open(f,
"w"){|o|o<<c.f(p,f,1)}}}||(DRb.start_service U,C.new{def f(c,a=[],t=2)c==x(U)&&(
t==0&&D[s(a)]||t==1&&F.read(s(a))||p(a))end;def y()(p(U)+p).each{|u|c(u).f(x(u),
p(U))rescue()};self;end;private;def p(x=[]);O.push(*x).uniq!;O;end}.new.y;sleep)

I think I've had more than enough of this. Pick a suitably high-level language, use single character variable names and some whacky formatting to exchange a file over a socket and call it P2P. Next someone is gonna write a java programme using SUN's JXTA that just inits a class or two, format it all on about 4 lines and say, wow it's the shortest P2P app ever... More interesting would be a P2P application written in a declarative language like Prolog or a functional language like Haskell or Hope. Haven't done much Haskell programming in a while (damn rusty and for some reason don't feel like breaking out the books) but prolog looks tempting. Expect a post. I may have to use some file IO/socket programming but it sounds like an interesting project. I'll let readers know how I get on ;-)

A friend (honest) forgot their pwd recently and asked me to hack into their machine and change their admin pwd. I found the following really tasty application which does the trick.
The Offline Password and NT registry editor by Petter Nordahl-Haggen. This is a very useful utility which I've used in the past and which has proven very effective. There are bootable floppy and CD images on the site that you can use edit your windoze box's passwords, stored in the reg's SAM file. For more hints and tips check here. Merry Christmas and a happy new year to everyone...

Just read this over on the reg. It looks like Sharman & the music industry (ARIA or the Australian Recording Industry Association in this case) have finally agreed what information from the Feb 2004 data seizures can be used in court. I know this is a complex issue and file-sharing networks don't necessarily have to be used for illegal purposes but ask yourself the following question:

Besides illegal copies of music and software what else do you look for on P2P filesharing networks?.

Personally I think P2P technology is fantastic. The self-organising and self-healing nature of many of these networks is a percursor for the next generation of internet architecture & design. As autonomics becomes an important field of IT research in the future P2P networks such as Free Net and Bit-Torrent will be recognised as a true paradigm-shift, to use that corny and overused phrase. Also the capabilities of a network such as OceanStore are phenomenal. There's a real pioneering spirit there and it's wonderful to be a small part of it. It's also unfair for the music industry to blame technologists when they're clearly trying to protect their market in the face of cheap/free technology for recording and distribution. Perhaps, they've forgotten that it was a series of technological evolutions and revolutions that led to the creation of the industry in the first place. Thank you Edison, Dolby, Philips Electronics et al. Technological change is an inevitable and those that don't move with it become industrial dinosaurs whose SEC filings are dug up by corporate paleontologists in the future as they try to piece together "What ever happened to that company industry?" So how will the industry evolve? I'm not sure but here's some plausible statements:

• Media distribution and playback technologies will be all-digital within the next 10 years


• DRM cryptographers and cryptologists will keep pace with eachother, leading to a stalemate situation where ever more draconian DRM initiatives are met with ever more sophisticated mechanisms for by- passing them


• The availability of a free and relatively unmonitored distribution channel (the internet) will ensure that illegal media distribution does not die out. Normally law-abiding citizens will simply 'opt out' of legitimate media distribution schemes.


• Many emerging artists will directly sell media over the internet for very low prices, cutting out the industry middleman


• Major artists who've already made enough money will embrace internet distribution as a way to gain greater artistic freedom


• Media costs will tumble as a result. A combination of trade agreements (European Community and Worldwide) and a cheap distribution channel, will standardise media costs so music, films etc. are available for the same basic price throughout the world


• Firesharing networks will continue to be popular and will resist attempts to shut them down. Americans will come to view secure and anonymous information distribution technologies as they currently do guns control. Laws that curtail the availability and use of those technologies will be viewed as undemocratic and an impediment to free speech.


• The AOL/Time Warner merger will finally start to make sense as such a conglomerate could recoup distribution costs through it's ownership of both the media rights and the distribution channel


• Apple's flat rate iTunes service will be the model for music distribution services for years to come


• Internet and Mobile Network Service Provider billing systems will become much more sophisticated to allow for differentiated service charges and itemised content billing.


• A beleaguered media industry will successfully lobby governments for other mechanisms for revenue generation, probably leading to direct media taxation based on media bandwidth usage


• In the future everybody will be a published artist
  

Feel free to comment...

Super 3G is the new faster than fast, whiter than white, icecold, hot as hell and totally groovy wireless transmission standard which is going to provide wireless speeds 10 times faster than 3G. The only catch is that it's not even a specification at the moment, rather an initial agreement between the major mobile operators and phone companies to develop a new services by 2009 (with technology optimistically concluded by 2007). I'll believe that when I see it but I suppose much of the ground work has been laid out in the latest releases of the UMTS standard. This service should enable all kinds of exotic scenarions like seamless roaming of continuous video feeds and the like. WOW! The reg has more information and gizmodo remarks on it as does ITWEB

The half-empty cynic in me still wonders whether this investment in hugely expensive new transmission technology is justified given that
"most wireless operators with 3G licenses have not made a profit on their high-speed wireless services after spending billions on licenses
Still all is not lost as according to the reg...

"In the nearer term, UK-based mobile operator mmO2 said in early December that it will launch a "super-fast" 3G network in 2005, promising speeds faster than current fixed broadband levels. The mobile operator, which has yet to launch its main consumer 3G service, said it will offer high-speed downlink packet access (HSDPA) technology and Internet Protocol Multimedia Services (IMS) with download speeds of 3.6Mbps, which will increase to 14.4Mbps as handsets become available."

Everybody interested in the future of digital media and the computer industry in general should have a look at the following flash movie. Available from broom.org. It's fascinating. I think that it's ultimately flawed in some of its suppositions about MAD (Mergers, Acquitisions and Divestitures)New York Times ®. I suspect that it's more likely that a merger or joint-venture between old-world and new-world media will be part of this revolution. Rather than bypassing established media companies and brands. Old media and new media will combine with a different, more cooperative organisational structure. Perhaps in 2010 computers will be able to provide the value-added interpretations that news companies currently provided. More thoughts on this later but I think that more sophisticated content licensing, canonical information addressing & DRM will play a part. Still, this really made me think and it's beautiful to watch.

• Several highly cautious owners


• Increasing revenues <(up 3.5 % to 26.5 million euro)


• Profitable (8.9 million euro in 2004)


• Large Customer base (340,000 subscribers)


• Lots of untapped potential for product diversification


• Knockdown price (< 20 % of 1999 purchase prices of 680 million euro)

Excellent article from Martin Fowler about the dangers of using metaphoric reasoning when comparing professions. To quote Martin

... it all comes down to how you use the metaphor. Comparing to another activity is useful if it helps you formulate questions, it's dangerous when you use it to justify answers.

I'm supposed to be working on documents today but I must admit that I've got other things on my mind. One of these is how to turn my microwave oven into a personal foundary. I have a hankering for another japanese ceremonial blade (a nice new katana). but I haven't figured out how to smelt in my house yet. However home based metallurgy is being brought a giant step closer using the Reid Technique (RT)

"a simplified ceramic-shell procedure for the casting of non-ferrous metals, patented in 1990. RT was first developed to avoid the problem of heat loss, which makes the the pouring of small melts very difficult - these difficulties arise however the metal is heated, and while the microwave technique set out here can be used for heating small amounts of metal in open crucibles, its greatest potential lies in its use as a flameless furnace in processes such as the Reid Technique. The crucial discovery, made during extended tests with various susceptors - materials which heat up when exposed to microwaves - was that two substances, graphite and magnetite, working together were required to achieve the kind of heating we were looking for."

Sometimes stock control & point-of-sale systems need to generate barcodes. Xyling java blog has a recent article on how to achieve this in your JFC and J2EE applications. This software is licensed under the Lesser GNU Public License (LGPL). JBarCodeBean supports all major barcode formats including:

• Code 128
• Code 128
• Code 39
• Extended Code 39
• Codabar
• Interleaved Code 25
• MSI
• EAN-13
• EAN-8

I got thinking about the implications of IM-Bots when I worked for the TSSG research group in Ireland. IM-Bots are pseudo-intelligent automated IM buddys that responds to queries using greater or lesser degrees of Natural Language Parsing (NLP) and knowledge inference. One of the most famous is the Alice-Bot which uses Artificial Intelligence Markup Language (AIML) to associate semantics (like running a program) with parsed syntax. I've written a paper about using IM-Bots as virtual shopping assistants which is available here

Kuro5hin has a neat article today about mechanisms for combating Referrer Spam. This kind of Spamming involves hitting websites while facking the referrer info in the HTTP request. Often referrer information is publically or privately available through webserver log analyser packages such as webalizer. However, even if you're not worried about pollution of your weblogs and published usage stats Referrer Spammers have an irritating habit of DDoSing your site into oblivion while they taint your logs. Charming people I'm sure.
One of their suggested methods for blocking these wonderful folks involves blocking their URLs using your .htaccess file. I use the following voodoo with my movable type weblog and it's very effective.

SetEnvIfNoCase Referer ".*(credit|texas-hold-em|holdem|viagra|sex|more-naughty-words).*" BadReferrer
order deny,allow
deny from env=BadReferrer

The process of maintaining up-to-the-minute blacklists in your .htaccess files can be automated using the catchily titled Referrer SPAM FUCKER 3000. Quality code that does exactly what it says on the tin ;-)
Hasta la vista texas-hold-em. Just trying to plant a few seeds here and help reclaim the internet from these Grade A 1u53r5

I seriously doubt it however!
iBill or "Internet Billing" is a secure online payments provider that promises to simplify the process of payments processing for small businesses that don't want the hassle of setting up their own merchant account or deploying their own online credit-card payment system. Sounds like a good idea & it is. However, iBill charge an absolutely staggering 30% of gross sales for this service. 20% processing charge and an additional 10% for "reserves". "That's amoral" I hear you say.. NO, this is amoral. It turns out that iBill haven't been paying their customers for months. With a wait of around 4 months for customers to find a new payment processor iBill can afford to make empty promises for quite a while, god knows where the money is going. According to the NYTimes article the company is currently under investigation by the US Dept of Justice & a class action lawsuit by disgruntled clients is reportedly on the way.
Interestingly enough I asked around & several friends thought that iBill was associated with Apple in some shape or form. the lower case i is very reminiscent of Apple's product nomenclature. Apple take note!

Pulled this from wired news. The difficulty of providing accomodation quickly and cheaply under arduous circumstances such as wars & natural disasters is a serious issue for governments, armies and aid agencies. According to Wired a solution could be at hand.

A pair of engineers in London have come up with a "building in a bag" -- a sack of cement-impregnated fabric. To erect the structure, all you have to do is add water to the bag and inflate it with air. Twelve hours later the Nissen-shaped shelter is dried out and ready for use.

The inventors filed a patent, which covers the concept of creating structures using a cement-impregnated cloth bonded to an inflatable inner surface. Full-scale production is planned and could take off soon, as Concrete Canvas is short-listed for the New Business Challenge run by Imperial College London and the Tanaka Business School. The winner of the £25,000 ($48,000) prize will be announced next week.

Not exactly a shock but indicative of both technical intent and direction at Microsoft. The full article is available at Grid Computing Planet. Most followers of P2P and collaboration technologies are familiar with Groove Networks offering. It enables the creation of a virtual office by connecting PC's together over a P2P network to create a secure workspace where document, applications and communications can be shared. The Groove software will be integrated within Microsoft Office, integrating with the RTC (Real-time Collaboration) s/w that Microsoft already bundles including Office Live Update.
In another twist, Ray Ozzie, the creator and CTO of Groove will become the CTO of Mc$oft's collaboration s/w division.

Ozzie, who will become CTO and report to Microsoft Chief Software Architect Bill Gates, is renowned for creating Lotus Notes, now a multi-billion-dollar business for IBM. He is also one of seven "Windows pioneers," an honor bestowed to engineers who have contributed to the improvement of the operating system.

Gates, who joined the conference call from Redmond, that Groove technology has "fantastic and very unique" properties Microsoft wants to put into Office.

"He's made a huge contribution in terms of giving us feedback about the platform," Gates said, citing Ozzie's work on Windows user interfaces and Visual Studio. "It's very exciting to have Ray and his team joining Microsoft. I think it's really going to help us do a better job for all of the information workers out there."

Noting that he has had the privilege of working with Ozzie for "many decades" as he was building applications on DOS or Windows, Gates said he had wanted to hire Ozzie for a long time. The CTO will have a great deal of say over corporate communication and collaboration offerings.

He will also continue his work with Groove's roughly 200 employees, which will remain at Groove's Beverly, Mass., headquarters as part of Microsoft's Information Worker Group.

Ozzie said his plan for Groove reflected how the business environment was changing, including a different type of security model based less on setting up firewalls and more on how people work with one another.

"Over the years, we've been very fortunate have led us all to be carrying around Wi-Fi-enabled laptops," Ozzie said. "The nature of work itself has changed for many of us. We very commonly do work in a geographically dispersed fashion, in the office, at home, in hotels, at Starbucks and so on. Our interactions involve being on multiple networks..."

Microsoft had already demonstrated its committment to both Groove Networks and collaboration software with a $38 million investment in Groove during 2003.

Both are two subjects that I'm becoming very interested in recently. In trying to develop a clear understanding of all the implications of National (Irish), European and World Market (US, predominantly) data retention legislation I've been having a look at the Irish Data Protection Act. More information is available at the authoritative DataProtection Commissioner's website. However, the following point struck me.
Section 2 of the original 1988 Act (which is still valid AFAIK) states that

A data controller should observe certain principles in relation to personal data:

• The data or information constituting the data shall be obtained and processed fairly


• the data shall be accurate and where necessary kept up to date.


• Data held for back-up purposes is exempt


• shall be kept for one or more specified or lawful purposes - specified refers generally to purposes specified in any registration document, where applicable


• shall not be used or disclosed in any manner incompatible with such purpose(s)


• shall be adequate, relevant and not excessive in relation to that purpose(s)


• shall be kept for no longer than is necessary; data held for historical, statistical or research purposes is exempt.

I posted not so long ago about Georgia Tech's prescient presentation on the media museum of 2014. While reading the sunday dailies & the sunday blogs today I noticed that the observer has gotten a shiny new daily blog. Here's the low down from boing-boing

The weekend paper is now supplemented by a daily blog, with podcasts and moblogs. The RSS is fulltext (crap, no it's not -- this is such an important detail, Observer -- get it right!). Trackbacks and comments are on and unmoderated. Keywords are tracked and displayed in a "folksonomic zeitgeist." Headlines from competing papers and Technorati link cosmoses are pulled in and displayed on the front page. No paywall. No adwall. No wall.

Just picked this up on Electric News today. An English company called Colt Telecom are offering irish businesses all local, national and international calls within a select number of countries for a flat rate, per-user, per-month fee of EUR24.50. Other countries covered in the deal include Austria, Belgium, France, Denmark, Germany, Italy, Netherlands, Portugal, Spain, Sweden, Switzerland and the UK.

However, the monthly charge does not cover calls to mobile phones. Having just read the VoIP open access directions from Comreg I'm not surprised as the call tariffs & STRPL (Switched Transit Routing Price List) for voice to mobile are likely to be at mobile-to-fixed rates for VoIP calls so as not to penalise the impoverished mobile operators, who've got to make a buck too, ya know.
It looks like Colt will be using IP-Centrex for switching and customers will lease phones at a flat rate of 20 euro/month. Nice idea as it reduces business CAPEX cost considerably while capturing a market for Colt. It appears that carrier-grade SIP-based Siemens HiPath 8000 softswitche(s) will form the switching backplane of the service and there are plans to introduce some unified messaging offerings which integrate with MS outlook. Not sure if a webservice only offering will be available to those that don't run windows.

Colt, which employs around 40 people in Ireland specialises in providing data, voice and managed services to midsize and major businesses and wholesale customers. It has more than 50,000 customers across all industry sectors and the company owns and operates a 13-country, 20,000km network that includes metropolitan area networks in 32 major European cities, with direct fibre connections into 10,000 buildings and 12 Colt data centres.

Yay, finally a post about technology. No science, religion or philosophy today. Hell, I'm not even going to consider the ethical or philosophical implications of having to pay for culture as opposed to it being provided free gratis to all. Still, Digital Rights Management (or DRM) has caught the imagination of an entertainment industry keen to avoid getting overtaken by the latest and greatest piracy technologies and the increasingly flexible morality of the general public. They wouldn't sell us a DVD writer if they didn't want us to copy movies, right???. I was extremely interested to pick the following article from gizmodo. It really does look like Real Network's "Rhapsody To Go" service just plain doesn't work. The technology is Real's implementation of the the Windows Media Player 10 DRM that lets you rent downloaded tracks and even listen to them on your portable music player. These so-called portable subscriptions are a really nice feature that fits with how most users would envisage DRM. You subscribe to the content and then enjoy it on the device of your choice! If the technology works that is. However PC World points out the many limitations of the service. You range of players that support it's DRM doesn't include the ubiquitous iPod. That's a bit like serving a vegetarian a steak sandwich. A bit pointless really! To make matters worse, PC World failed to successfully transfer media to any portable music player. Here's a quote from the article:

"In my tests, transferring tracks to a notebook and playing them while I was unconnected to the Net worked fine. Of course, that's no great accomplishment--other music services have been allowing something like that for years. But despite trying with two IRiver H10 MP3 players, two Rhapsody accounts, and two PCs, and getting suggestions from Real engineers, I was never able to transfer any Rhapsody track I hadn't bought outright onto a portable player. For me, at any rate, Rhapsody To Go just didn't work."

I'm just blog tired here :-D. Just looking at blogshares again. I'm slightly p*ssed off with the inaccuracies in it's valuation of my blog. I find it ridiculous that given the numbers of people that use the bloglines blog aggregator that the blogshares spiders can't bypass the single level of indirection implied by their javascript blogroll. It's a real pain in the ass as I'm damned if I'm going to do any server-side scripting to dynamically embed my roll in the page. Bloglines was recently acquired by AskJeeves which also owns the teoma patented thematic search technology. The idea appears to be the creation of a search engine that is capable of thematically searching and grouping a massive database of RSS blog and news feeds. Which is a pretty nice idea in my opinion. I wish them luck.

I've recently bought the rather snazzy Nokia 9300 smart phone. This is the tool-de-force (sic) for the roaming sys admin combining features like bluetooth & tri-band with a usable qwerty keyboard. In my opinion it's more attractive, compact and lighter than my trusty XDA so all-in-all I'm very happy with my new purchase. However I need to find a new home for my XDA-2. Rather than leave it gather dust I'm selling my 10 month old model in it's original packaging and including a bluetooth headset for around 400 euro. Sensible lower offers will be considered. Drop me a mail if you're interested.