25 September
2009
Building a Chumby-style feed reader using SJS
Stratified
JavaScript (SJS) might still be work in progress, but that doesn't mean
it can't be used for building some (semi-)useful things.
Consider e.g.
the 'FeedChumby' RSS feed reader which automatically cycles through the posts
of some popular RSS feeds, 'slide-show' style - similar to the way in which a Chumby cycles through its widget
stream.
In this blog post I will dissect the code for this webapp,
tutorial-style.
Accessing feeds
At its core, FeedChumby uses the Google AJAX Feeds API
which allows us to retrieve RSS feeds without having to worry about
any crossdomain issues and without requiring any server support from
our side.
SJS gives us access to this API through the unassuming function
Ajax.Google.Feeds.loadFeed(url);
which returns a JSON results object as described in the Google documentation.
Because we're using SJS, we can call this function synchronously - no
callback or other asynchronous construct needed. All the asynchronous
action is automatically choreographed behind the scenes: loading the
Google API if it hasn't been loaded yet, making sure the 'feeds'
module is loaded, and finally loading the actual feed itself. While
all of this is going on, the browser stays perfectly responsive.
Coupled with SJS's 'Ajax.DOM.waitForEvent' library function (which -
as one would expect from the name - waits for a particular DOM event),
loading and displaying a feed in response to a button click is then as
easy as this:
Ajax.DOM.waitForEvent("click", button);
var result = Ajax.Google.Feeds.loadFeed(feed_url);
renderResult(result);
Here is a complete example: google-feeds.html
Basic program structure
By sticking 'loadFeed' into a loop, iterating over a few different
feeds and over individual feed posts, and pausing for a few seconds
after each post is displayed (using SJS's "hold()" primitive), we've
got the basic structure for our Chumby-style feed reader - again no
asynchronous constructs (like e.g. setTimeout) needed:
var feeds = [ "...slashdot...", "...digg...", "...elreg...", ... ];
while (true) {
for (var i=0; i<feeds.length; ++i) {
var result = Ajax.Google.Feeds.loadFeed(feed[i]);
for (var j=0; j<result.feed.entries.length; ++j) {
renderFeedEntry(result.feed.entries[j]);
hold(10000); // <-- wait 10s before continuing with next post
}
}
}
Let's also add a button that allows the user to shortcut through the
10s waiting time. The easiest way to provide this sort of
functionality is by using SJS's '@' combinator (as described in an earlier blog post, "A @ B" executes both A
and B in parallel, and returns as soon as either of them have finished
evaluating; at the same time cancelling the still pending expression):
for (...) {
...
hold(10000) @ Ajax.DOM.waitForEvent("click", nextButton);
}
As a little refinement, we can replace the 'hold(1000)' by code that
will provide the user with some UI feedback during the waiting period:
function countdown() {
for (var i=0; i>0; --i) {
counter.innerHTML = i;
hold(1000);
}
}
...
for (...) {
...
countdown() @ Ajax.DOM.waitForEvent("click", nextButton);
}
Here is a complete example with the functionality so far: feedchumby.html
Adding 'pause' functionality
Again using '@' and 'Ajax.DOM.waitForEvent()', we can modify our
countdown() function to stop counting down while in a paused state:
var paused = false;
function countdown() {
var i=10;
while (i>0) {
counter.innerHTML = i;
if (paused) {
pauseButton.innerHTML = "Play";
Ajax.DOM.waitForEvent("click", pauseButton);
paused = false;
pauseButton.innerHTML = "Pause";
}
(hold(1000), --i) @
(Ajax.DOM.waitForEvent("click", pauseButton), paused=true);
}
}
Note the '@' expression here. It combines the two simpler expressions
"(hold(1000), --i)", which waits for 1s and then decrements 'i', and
"(Ajax.DOM.waitForEvent("click", pauseButton), paused=true)", which
waits for a click on 'pauseButton' and then sets the variable 'paused'
to true. (In both 'normal' JavaScript and also SJS, for two expressions
A and B, "A,B" is an expression that means "execute first A and then
B".)
Here is the example with the pause functionality rolled in: feedchumby2.html
Adding birectional navigation
To add functionality that allows the user to navigate back to the
previous post, we need to change our loop logic a little bit to
account for the situtation where we're moving past the beginning of a
feed to the previous feed. The gory details are here: feedchumby3.html.
Note the '@' expression in the main loop:
(countdown(), ++j) @
(Ajax.DOM.waitForEvent("click", "next"), ++j) @
(Ajax.DOM.waitForEvent("click", "previous"), --j);
Previously we only waited for the countdown to finish or for the user
to click on the 'next' button. Now we're also waiting for clicks on
the 'previous' button, and we're incrementing/decrementing variable
'j' to communicate the direction of navigation.
More embellishments
The absence of any asynchronous constructs (callbacks, setTimeouts,
...) makes the code quite modular. It's easy to add little features
here and there without having to take a "global view" of the
application, and the '@' combinator relieves us from a great deal of
manual bookkeeping.
The final version of the FeedChumby adds a couple more small touches,
like e.g. an animated pause button, and it contains some CSS to make
the application more presentable:
feedchumby4.html
I think FeedChumby is a pretty good example of the power of SJS for writing
Ajax-style applications. The real win, however, will only show in big
'mashup' style applications that access several different APIs
concurrently. Expect to see more demos coming along with APIs such as
the Google Translation API, Google Contacts, Facebook Connect, and others.
Posted by
alex at
11:41
|
Comments (0)
10 September
2009
Stratified JavaScript and Multitasking
In response to my earlier blog post on Stratified JavaScript, Jonathan
Cook asked an interesting question:
"[...] I am very interested in using SJS to eliminate the blocking
nature of heavy computation in nested loops.
A hold(0) would seem to do this, or perhaps a different method for
this purpose would be appropriate?"
The short answer is, you don't need to call hold() or do anything
special at all. Just code your computation as you would in 'normal'
JavaScript, and SJS will take care of the rest; the UI will
automatically stay responsive while your computation is chugging
along.
Now for the slightly longer answer.
The SJS runtime contains a scheduler which preemptively schedules the
different strands of execution ('strata') of an SJS program. This
scheduler automatically yields control to the browser every so often,
and it ensures that individual strata get a fair timeslice.
In other words, SJS implements a form of preemptive multitasking,
rather than cooperative multitasking. There is no need to manually
yield control like you would in e.g. Narrative
JavaScript/Strands; SJS automatically does it in the background.
Here's an example of a heavy computation in SJS; a program to calculate a
few hundred digits of Pi:
pi.html
There are a couple of noteworthy things about this program:
Firstly, if you look at the source, there is no hold() or any other
special SJS primitive. It is just 'straight' JS code, which is being
preprocessed through the SJS compiler by virtue of sitting in a script
tag with type "text/sjs".
Secondly, it runs painfully slow. This is a consequence of the fact
that the SJS compiler doesn't perform any optimizations at all at this
point; every JavaScript function call or operator application is
converted into a potential yielding point, adding huge overhead to
programs which are heavy on mathematical calculations.
This situation will improve dramatically when we get around to making
the compiler smarter. A similar optimization in
Oni resulted in a 2 orders of magnitude improvement!
In the meanwhile there is a workaround though: SJS code can freely
call 'normal' JavaScript code, so we can just move critical inner
loops into 'normal' JavaScript, as in this modified program:
pi-faster.html
You'll notice that this program runs a lot faster, while not
really adding much complexity - just one inner loop has been moved to
'straight' JavaScript.
Just to prove that the preemptive scheduler works as advertised, here
are another couple of examples which add some SJS-specific features.
Firstly, this program adds an animation running in parallel to the Pi calculation:
pi-and-animation.html
Secondly, and finally, this program adds a 'UI' for turning the animation on and off:
pi-and-animation-and-events.html
Take a look at the animation loop:
function animateLoop() {
var elem = document.getElementById("toggle_animation");
while (true) {
waitForEvent("click", elem);
elem.innerHTML = "Stop animation";
animate() @ waitForEvent("click", elem);
elem.innerHTML = "Restart animation";
}
}
Isn't it beautiful? :-)
Posted by
alex at
15:52
|
Comments (4)
27 August
2009
Introducing Stratified JavaScript
Here's a sneak preview of a project I've been working on in my spare
time for a while, and which is nearing its first release:
'Stratified JavaScript' ('SJS') is a cross-browser extension to
JavaScript which adds some concurrency features to the
language. Without going into too much detail, the main features
include:
(1) The ability to pause execution
The function
hold(time_ms);
pauses execution of a piece of code for time_ms milliseconds. E.g.:
var elem = document.getElementById("animated_element");
var x = 0;
while (true) {
elem.style.left = x;
x = (x + 10) % 200;
hold(100);
}
(full sample:
animate.html)
Note that 'hold' doesn't busy wait. The browser's UI stays fully
functionally while this code executes.
(2) Fork-Join Parallelism
In conventional JavaScript, arguments to function calls are evaluated
sequentially. In Stratified JavaScript, they are evaluated concurrently:
function animate(elemname, duration_ms, step_ms) {
var elem = document.getElementById(elemname);
var start_time = new Date();
var x = 0;
do {
elem.style.left = x;
x = (x + 10) % 200;
hold(step_ms);
} while (duration_ms > new Date() - start_time);
}
function par(a, b) {
alert("all done");
}
par(animate("animated_element_1", 10000, 100),
animate("animated_element_2", 7000, 80));
(full sample:
forkjoin.html)
Here, the two 'animate' calls are being executed concurrently. Once they
both return, the body of 'par' gets called.
(3) Exploratory parallelism
Many concurrency problems fit into the pattern
"Explore options a,b,c,... concurrently. Once one of them yields a satisfactory
result, return it and abort exploration of the remaining options."
Exploratory parallelism in Stratified JavaScript is embodied by the
'@' ('alt', 'alternatives') operator:
animate("animated_element_1", 10000, 100) @
animate("animated_element_2", 7000, 80);
alert("all done");
(full sample:
alt.html)
This code pops up 'all done' after 7s (after the animation of
animated_element_2 has finished). At the same time, the animation of
animated_element_1 will be aborted.
In many ways, exploratory parallelism forms the heart of Stratified
JavaScript. It is applicable to many situations which are very
cumbersome to express in conventional thread-based parallelism.
(4) Suspend/Resume
Most JavaScript programs live by asynchronous events, e.g. events
generated by button clicks, or asynchronous
XMLHttpRequests. Stratified JavaScript contains a generic mechanism
for converting these asynchronous constructs into synchronous ones:
suspend {
... resume() ...
}
retract {
...
}
finally {
...
}
This will be executed by first evaluating the code in the 'suspend'
block and then suspending execution. Execution will resume when the
'resume' function defined in the suspend block is called. The optional
'retract' block will be executed if the current code is aborted before
'resume' was called. The optional 'finally' block will be executed in
either case; if execution was resumed, or if execution was aborted.
E.g., this is how an event listener could be 'synchronized':
function waitForEvent(event, elem) {
suspend {
var rv;
var listener_func = function(e) {
rv = e;
resume();
};
elem.addEventListener(event, listener_func, false);
}
finally {
elem.removeEventListener(event, listener_func, false);
}
return rv;
}
With this function we can now e.g. wait for a button click:
while (true) {
waitForEvent("click", document.getElementById('button1'));
alert("You clicked the button");
}
(full sample:
suspend.html)
Or something more complicated:
function dump(message) {
document.getElementById("output").innerHTML = message;
}
while (true) {
var e = waitForEvent("click", document.getElementById('button1')) @
waitForEvent("click", document.getElementById('button2')) @
hold(5000);
if (e)
dump("You clicked button '"+e.target.id+"'");
else
dump("Click a button already!");
}
(full sample:
suspend2.html)
If all of this sounds similar to Oni, that's because it is. Under the hood, Stratified JavaScript is executed by a runtime based on Oni.
Everything that can be expressed in Stratified JavaScript can be written directly in Oni without going through any compilation stage. The advantage of Stratified JavaScript is that
it doesn't have the awkward functional syntax of Oni.
Posted by
alex at
18:25
|
Comments (7)
27 April
2009
Ubiquity, Oni, and Composability
Ubiquity and Composability
One case where the limitations of conventional asynchronous code
bubble to the surface is in systems
like
Ubiquity. E.g. Ubiquity
has a 'google' command and a 'translate' command. I can instruct
Ubiquity to google for something:
google foo
and I can get Ubiquity to translate something:
translate The quick brown fox to German
but I cannot instruct it to give me list of translated google results:
translate (google foo) to German // doesn't work
Or email me the resulting list:
email(translate (google foo) to German) // doesn't work
Or email me a list of laptops on ebay below some price:
email(filter(price<500dollars, find_on_ebay(macbook)) // doesn't work
(Ok, excuse the syntax for these examples; in 'real-world' Ubiquity
you would probably want this to be in a more natural language style,
but I hope I'm getting across the idea of composability. Or rather:
lack thereof.)
Oni and Composability
So how does Oni relate to
this? Oni is a
browser-based "embedded structured concurrency framework". It allows
you to write asynchronous code as if it was synchronous, adding back
the kind-of composibility that is lost when juggling concurrent strands of execution (such as e.g. pending XMLHttpRequests) with 'conventional' sequential languages.
Oni has recently gained a small AJAX library with some bindings to the
Google AJAX APIs. With this new functionality we can write some
semi-useful Oni programs which nicely demonstrate the type of
composability that Oni provides.
One of the new Oni functions is Google.Feeds.Load(url), which
retrieves a JSON object of the RSS feed at 'url'.
With this function it is easy to write an Oni program that e.g.
displays the most recent item posted to planet.mozilla.org:
Display(GetMostRecentPost("http://planet.mozilla.org/atom.xml"))
(full sample:
composability1.html)
where Display, and GetMostRecentPost are implemented as follows:
var Display = SLift(function(html) {
document.getElementById('output').innerHTML += html;
});
'SLift' is an operator that 'lifts' a conventional sequential JavaScript
function into Oni.
var GetMostRecentPost = Defun(['url'],
ObjMem(Google.Feeds.Load(Get('url')),
'entries', 0, 'content'));
'GetMostPost' is a new Oni function which uses 'Google.Feeds.Load' to
load the feed, and then traverses the returned feed JSON object using
ObjMem, to finally return the html content of the current first post.
So far so good, but this is hardly exciting. Let's add another function into the mix:
Google.Language.Translate(text, srclang, destlang), which returns a
translation of text (assumed to be in language 'srclang') to language
'destlang'. The object returned by this function is a JSON
'translation result' object of which we only want the 'translation'
member, so let's wrap it into a 'Translate' function which extracts
just this member:
var Translate =
Defun(['txt', 'lang'],
ObjMem(Google.Language.Translate(Get('txt'), 'en', Get('lang')),
'translation'));
Armed with this new function, we can now do something a little more
interesting, e.g. retrieve a feed entry and translate it into German:
Display(Translate(GetMostRecentPost("http://planet.mozilla.org/atom.xml"),
'de'))
(full sample:
composability2.html)
Or retrieve a post and display it in English, German, Spanish and Japanese:
Let({post:GetMostRecentPost("http://planet.mozilla.org/atom.xml")},
Par(Display(Get('post')),
Display(Translate(Get('post'), 'de')),
Display(Translate(Get('post'), 'es')),
Display(Translate(Get('post'), 'ja'))))
(full sample:
composability3.html)
Here, 'Let' introduces a new variable ('post'). 'Par' executes its
subexpressions in parallel, so the ordering of what's displayed on
screen will depend on the order in which the translation requests
return.
Here's another example: Fire off requests to get the most recent post
from slashdot and the most recent post from planet mozilla and display a
translated version of the first one that comes in:
Display(Translate(Alt(GetMostRecentPost("http://rss.slashdot.org/..."),
GetMostRecentPost("http://planet.mozilla.org/...")),
'de'))
(full sample:
composibility4.html)
Here, 'Alt' is an Oni operator that executes its subexpressions in
parallel and returns the first one that returns, at the same time
cancelling the other one.
And for a final example, let's add a timeout to the previous
example. If we don't get a translation of a planet or slashdot post
within 1 second, cancel any pending requests and display a message instead:
Display(Alt(Translate(Alt(GetMostRecentPost("http://rss.slashdot.org/..."),
GetMostRecentPost("http://planet.mozilla.org/...")),
'de'),
Delay(1000, "Sites timed out")))
(full sample:
composibility5.html)
So what's the big deal?
What these examples illustrate is composability, something that
the conventional way of dealing with concurrency lacks. The 'standard'
way of coding the examples above would be by using callback
functions. This works well enough for simple cases, but as programs
become more complex you soon enough end up in 'callback hell'. In Oni
(and similar systems such
as Orc or
Arrowlets),
we can build more
complex programs out of simpler ones in a modular, structured fashion.
So what's the relationship to Ubiquity? Oni is a low-level framework layered on top
of JavaScript. It is not, and does not aspire to be a human interface
in the way that Ubiquity is. The relation between the two is only
tangential: I believe that Ubiquity could become much more expressive
if layered on top of a system like Oni.
If this got you interested, and you'd like to find out more about Oni,
please visit the project homepage
at http://www.croczilla.com/oni.
Posted by
alex at
13:02
|
Comments (1)