August 2007 Archives

I stumbled across this fun page today. I think my favorite codepoint name of those listed is U+2764 "Heavy Black Heart"...

I doesn't look like that page is actively maintained any more, so I don't think you can submit your own favorites.

Here's how to use the new Fiber class (warning: class name may change) to generate an infinite sequence of Fibonacci numbers. I use "generate" in the sense of Python's generators. Ruby's new fibers are "semi-coroutines"

fib = Fiber.new do 
  x, y = 0, 1
  loop do 
    Fiber.yield y
    x,y = y,x+y
  end
end

20.times { puts fib.resume }

Remember that the Fiber class was added to Ruby 1.9 yesterday, so you need a very recent snapshot to make this work.

Here's a slightly higher-level way to do the Fibonacci numbers. We hide the Fiber class inside of a Generator class:

class Generator
  def initialize &block
    @f = Fiber.new &block
  end

  def next?
    @f.alive?
  end

  def next(*args)
    @f.resume(*args)
  end
end

fib2 = Generator.new do 
  x, y = 0, 1
  loop do 
    Fiber.yield y
    x,y = y,x+y
  end
end
20.times { puts fib2.next }

As a followup to the post below, here is some example code using fibers to implement co-routines. This works in today's snapshot build, but it uses brand-new stuff, so it probably won't work if your build is more than a day old. Similarly, the API may change again, so it might not work if your snapshot is too new, either. svn.ruby-lang.org seems to be down (for me at least) right now, so I got my snapshot here: http://www.mirrorservice.org/sites/ftp.ruby-lang.org/pub/ruby/snapshot.tar.gz The code works for me on Linux, but I haven't tested it elsewhere:

f = g = nil

f = Fiber::Core.new { |x|
  puts "F1: #{x}"        
  x = g.transfer(x+1)    
  puts "F2: #{x}"
  x = g.transfer(x+1)
  puts "F3: #{x}"
}

g = Fiber::Core.new { |x|   
  puts "G1: #{x}"           
  x = f.transfer(x+1)
  puts "G2: #{x}"
  x = f.transfer(x+1)
}

f.transfer(100)

This code prints the following

F1: 100
G1: 101
F2: 102
G2: 103
F3: 104

Notice that the code uses the class Fiber::Core, and its instance method transfer. There is also a Fiber class, with a slightly higher-level API involving the instance method resume and the class method yield. I'm not sure exactly how these are intended to be used, but I was able to produce the same output as above using Fiber (instead of Fiber::Core) as follows:

g = Fiber.new { |x|
  puts "G1: #{x}"
  x = Fiber.yield(x+1)   # Can't use resume here: double resume error
  puts "G2: #{x}"
  x = Fiber.yield(x+1)
}

f = Fiber.new { |x|
  puts "F1: #{x}"
  x = g.resume(x+1)
  puts "F2: #{x}"
  x = g.resume(x+1)
  puts "F3: #{x}"
}

f.resume(100)

Update: Koichi SASADA notes on ruby-core:

Fiber::Core and Fiber::Core#transfer is black magic. So I'm planning to rename this class and method to Fiber::DangerousCore::__unsafe_transfer__I_cant_promise_your_program_run_correctly__!

He also says that the Fiber class is a "semi-coroutine" like Python's generator and that the method names Fiber.resume and Fiber::yield are taken from the language Lua. Time to go read up on Lua, I suppose

Ruby 1.9 has, for a while now, supported Enumerators. These are Enumerable objects returned by iterator methods when the methods are invoked with no blocks. Until recently, Enumerators were still internal, Ruby-style iterators that you invoke with each.

But that has changed. I haven't seen this discussed anywhere in English, but recent builds of Ruby 1.9 (my build is from August 17th) also define methods next and next? on Enumerator. These are Python-style (and Java-style) external iterators:

Update: 8/24/07: Matz has just removed the next? method. So you now have to be ready to catch StopIteration when using Enumerators. This means that the twine method below doesn't work anymore..

r = 1..3   # An enumerable object
e = r.each  # An enumerator object
e.next   # => 1
e.next # => 2
e.next # => 3
e.next # StopIteration exception raised

True external iterators make the age-old problem of parallel iteration in Ruby trivial to solve. Here are some examples:

def twine(*enumerables)
  enumerators = enumerables.map { |x| x.each }
  while not enumerators.empty?
    e = enumerators.shift
    if e.next?
      yield e.next
      enumerators << e
    end
  end
end

def braid(*enumerables)
  enumerators = enumerables.map { |x| x.each }
  begin
    loop do
      values = enumerators.map {|x| x.next }
      yield *values
    end
  rescue StopIteration
    # This is normal termination condition
  end
end

a = [1,2,3]
b = [4,5,6]
twine(a,b,'a'..'b') { |x| puts x }
braid(a,b,7..10) { |x,y,z| print "#{x},#{y},#{z}\n" }

It appears that these new external iterators are not build on the old Generator library which used continuations (I think). Instead, they use a new Fiber class (a kind of micro-thread). It is not documented yet, but its in there in recent 1.9 builds. If you know how to use Fiber, feel free to explain in the comments.

This is a repost. The original from August 2nd started getting comment spam, and I accidentally deleted the post while deleting the spammy comments. The original, valid comments are gone as well, which is unfortunate, because Tom Klaasen and Dmitry Kim made helpful points that resulted in corrections to the post below.

Someone recently emailed the ruby-core mailing list asking "Is there some list of 'bullet points' on the major differences between the syntax of Ruby 1.8 and Ruby 1.9 available somewhere?" The response, of course was a link to the definitive list of changes in Ruby 1.9.

But that is an exhaustive list instead of just highlighting the major changes. So, the following is my somewhat more digested list of the important changes, as I understand them. Additions, corrections, clarifications, and so forth are welcome in the comments.

I recently learned about the Module.const_missing method for lazily computed constants, and came up with this Unicode utility module to try it out.

#
# This module lazily defines constants of the form Uxxxx for all Unicode
# codepoints from U0000 to U10FFFF. The value of each constant is the
# UTF-8 string for the codepoint.
# Examples:
#   copyright = Unicode::U00A9
#   euro = Unicode::U20AC
#   infinity = Unicode::U221E
#
module Unicode
  def self.const_missing(name)  
    # Check that the constant name is of the right form: U0000 to U10FFFF
    if name.to_s =~ /^U([0-9a-fA-F]{4,5}|10[0-9a-fA-F]{4})$/
      # Convert the codepoint to an immutable UTF-8 string,
      # define a real constant for that value and return the value
      const_set(name, [$1.to_i(16)].pack("U").freeze)
    else  # Raise an error for constants that are not Unicode.
      raise NameError, "Uninitialized constant: Unicode::#{name}"
    end
  end
end

I've always thought that java.util.concurrent.Exchanger was a nifty little class. The javadoc I've linked to describes it like this:

A synchronization point at which two threads can exchange objects. Each thread presents some object on entry to the exchange method, and receives the object presented by the other thread on return.

The javadoc also includes a nifty little example demonstrating why you might want an Exchanger object.

I decided to try to implement this in Ruby, to test my understanding of Mutex and ConditionVariable. The code is below. I'm not sure how useful it actually is, but it was fun to write. And if you can understand what it does, then you've got a good working knowledge of threads in Ruby!

Finally, if you've got the JDK installed, unzip the src.zip file and take a look at the source code for in java/util/concurrent/Exchanger.java. It is much more complicated than this trivial example. It was eye-opening to see how carefully written classes like this one are for high performance on multi-core or multi-CPU systems.

William Gibson's latest novel, Spook Country is out in hardback!

I'm not done yet, but I'm really enjoying it so far. An engaging story, plus he revisits/reconsiders some ideas from Neuromancer and Virtual Light

The first article of my new column on Java 7 is now up on java.net.

In the article I discuss the jdk7 and OpenJDK projects at java.net, mention some projects that are being developed openly and might become part of Java 7, and speculate that, because of schedule constraints, Java 7 might include a lot less than we were told to expect 10 months ago.

Future installments of the column will, I hope, go into more depth with specific APIs and will actually include code examples.

Some guy named John Richmond, apparently of Milford, Michigan, hacked the HTML form on the Paypal payment page for my Jude software, so that he could send me a payment of two cents instead of the $48 that I actually charge for the software. I validate the payment amount on the server side, of course, so my scripts never sent him a software license. Paypal took the two cents as a transation fee, I got nothing, and Mr. Richmond was, one assumes, two cents wiser.

But no, he had the gall to initiate a Paypal dispute demanding that I send him a license or refund his two cents. As part of the dispute process he wrote "I paid what the software was worth". He doesn't like my software so he uses the Paypal to send an insult and then uses the dispute process to yank my chain!

What a jerk!

In addition to griping, the point of this post, I guess, is to get something into Google, so that anyone considering doing business with John Richmond can find out about his jerkiness. Richmond uses the domain name clientsg.com, which is owned by his defunct Michigan corporation "Client Services Group, Inc.". Google shows listings for "Salepoint Inc." at the same address and phone number as "Client Services Group". Salepoint is not a Michigan corporation. Perhaps Richmond consults for salepoint.com, or perhaps he just uses that name for some other business.

Update: Paypal sent the guy his two cents back. So I guess the paypal user agreement allows strangers to send you money for no reason and then demand that you send it back.

Update 2: I'm getting tired of updating this post, but since I've been bashing Mr. Richmond, I should acknowledge a useful and unintentionally funny response I finally got from him. He writes:

Paypal allows you to use encrypted data in your HTML to accept payments.

You need to do this.

Contact paypal for the information on how to send your data as an encrypted block via a web page to paypal so idiots viewing the source code cant see it and modify it.

At least you check the payment amount. You would be amazed at how many sites there are out there that are automated and simply send out the licence when any payment amount is received.

The useful part: the info about encryption. Though since this is the only person who has attempted this in three years, I don't think I'll run out and patch my scripts now. The funny part: he acknowledges that he's an "idiot" :-)

Also, the timing of the comment from "me" strongly suggests that it is from Mr. Richmond himself. He seems to call himself "jobu", and he acknowledges that he exploits the weaknesses in sites Paypal scripts.