Musings

Instiki

There were some prerequisites.

MacOSX 10.4 comes with Ruby 1.8.2. But Instiki requires 1.8.4. Fink to the rescue:

fink install ruby
fink install sqlite3-rb18
fink install ruby18-dev

For reasons that will be clear in a moment, I also had to install SWIG 1.3.

Getting Instiki, itself, up and running was absurdly easy.

1. Download and unpack the latest source. (There’s a binary installer for an older MacOSX version; don’t bother.)
2. Type

   ./instiki

3. Connect to http://localhost:2500 and configure your wiki.
4. Done!

itex2MML

Next, I used SWIG to turn itex2MML into a Ruby extension¹. itex2MML 1.1.8 has two new make targets.

make
sudo make install

will, as before, create a stream filter usable by lots of different software, including my MovableType plugin. But now

make ruby
make test_ruby
sudo make install_ruby

will produce a Ruby extension. To use it, you do something like

require 'itextomml'

itex =  Itex2MML::Parser.new
itex.html_filter(string)

which return the filtered text.

Anyway, the procedure for building a Ruby extension using SWIG is a little bit platform-dependent. To accommodate the idiosyncrasies of different platforms, the Makefile relies on special features of GNU Make (the default version of make on Linux and MacOSX).

XHTML

Modifying Instiki to emit well-formed XHTML was pretty easy. (This is why I wanted to stay away from all those spaghetti-code PHP projects.) The entire patch file (including the code to create a Markdown+itex2MML filter) is a mere 1/7 the size of my MovableType patch file. Update: Please don’t use this. Instead, use this package, based on the current Instiki trunk. I can’t (and haven’t tried to) back-port all of my fixes to Instiki 0.11.

Of course, I don’t have anything nearly as bullet-proof. And I currently don’t even bother to do content-negotiation. Everyone gets application/xhtml+xml; since the Wiki is a private research tool, I don’t care about browsers that don’t support XHTML+MathML.Update: My version of Instiki now does Content-Negotiation, serving application/xhtml+xml to browsers that support it, including IE+MathPlayer.

Still, it bugs me that Markdown does little to ensure well-formedness, so it’s still painfully easy to screw up and produce a Yellow-Screen-of-Death. At some point, I’ll have to figure out how to hook in the Validator.

TODO

1. Either make the thing bullet-proof, or use a hack to serve MathML in a tex/html environment.
2. One of the features that appealed to me about Instiki was its “Export to TeX” feature. That doesn’t really work right (I don’t think the author was anticipating actual equations in the text.) But it could be fixed. Which would be very cool.
3. Since Instiki (by default, at least), runs in its own webserver, I can’t rely on Apache for access-control. You can publish multiple wikis (“Webs” is what Instiki calls them) under the same installation, and give each one its own password. But this has it precisely backwards. I don’t want to have to remember N passwords for N different wikis. I want to assign each person a password, and then use access-controls to determine what areas that person can access.

Probably there’s other stuff, but right now I should probably add some more notes…

Update (1/9/2007): Ruby Sucks

Having spent some more time with Instiki, I begin to see its limitations.

First of all, Ruby’s Regex parser is incredibly lame. If you search the web, you’ll find plenty of complaints about its stack overflowing should you happen to sneeze in its vicinity. In my case, BlueCloth would choke on a modest 3-row, 10-column XHTML table with the error:

Stack overflow in regexp matcher: /
	^				# Start of line
	<(p|div|h[1-6]|blockquote|pre|table|tr|td|dl|ol|ul|script) # Start tag: \2
	\b				# word break
	(.*\n)*?			# Any number of lines, minimal match
	<\/\1>				# Matching end tag
	[ ]*				# trailing spaces
	(?=\n+|\Z)			# End of line or document
	/ix

Moreover, if I tweaked BlueCloth to add <math> to the list of tags that it passes through unmolested (something that is standard in the Perl version of Markdown)

 BlockTags = %w[ p div h[1-6] blockquote pre table dl ol ul script math]

then all Hell would break loose, with stack overflows right and left.

I quickly retreated from that idea, though I did manage to accomodate the table by the following amusing hack. I disabled the special processing of HTML comments in BlueCloth:

  # Matching constructs for tokenizing X/HTML
  HTMLCommentRegexp  = %r{ <! ( -- .*? -- \s* )+ > }mx
  XMLProcInstRegexp  = %r{ <\? .*? \?> }mx
- MetaTag = Regexp::union( HTMLCommentRegexp, XMLProcInstRegexp )
+ MetaTag = XMLProcInstRegexp

  HTMLTagOpenRegexp  = %r{ < [a-z/!$] [^<>]* }mx
  HTMLTagCloseRegexp = %r{ > }x

and then inserted

<!--> Hack to work around Ruby's feeble Regex parser
</table>
<table>
<-->

inside the table, to break it into manageable chunks. That “works,” but is lame beyond belief.

More troublesome is that Instiki’s speed deteriorates drastically with the size of the page. Modest-sized pages are still very fast. But a more substantial page (19 KB Markdown+itex, 96 KB, after being processed by itex2MML.rb and 101 KB when rendered to XHTML+MathML by BlueCloth) takes nearly 5 minutes (!) to render. Once rendered, it’s kept in memory, and Instiki delivers it lightning-fast. But every edit of the page is painful.

The Perl version of Markdown (as used in MovableType) has no trouble processing such a page. I don’t think the problem is incompetence on the part of the authors of Instiki and BlueCloth. I think the problem is that Ruby sucks (at string processing).

Update (1/23/2007): Fixed!

Replacing the default BlueCloth by another Markdown implementation proved to be the solution. The wiki I’m now running is based on the development version of Instiki (which contains some essential improvements over 0.11) You can get it, either as a tarball, or by doing a bzr pull from my BZR repository.