AMQP

Sagittarius supports AMQP, Advanced Message Queue Protocol, since probably 0.6.x, I don't recall which version, to be honest. The reason I wanted to support this was because I was using QM, I think IBM MQ, at that moment at my work and wanted to send a message from a command-line, instead of using a Web browser. Unfortunately, the version of MQ was too old and didn't support AMQP yet. So, I couldn't use it for the purpose that I wanted. Nevertheless, it's there and it's supported.

Now, I want to write a demo application to show what Sagittarius can do out of the box. One of the main reasons I've made Sagittarius is that I can use it for daily commercial work. So, I thought it's nice to have a demo of integration with external systems, not only for HTTP but also some other protocols which Sagittarius supports out of the box. Then, I realised that the current implementation of AMQP client is not really efficient nor doesn't work as I expected.

To explain what doesn't work, I first need to explain how the AMQP connection and session work. An AMCP connection is a physical connection, which holds an actual socket. Now, under a connection, there are sessions. A session is a virtual separation of input and output channels. This means if a connection has multiple sessions, input or output messages need to be dispatched properly to the associated session. It's easier to see it in a diagram:

Session under connection

  Client App                                        Broker
+-------------+                                +-------------+
|             |################################|             |
|   +---+     |--------------------------------|    +---+    |
|   | C |     |            Session             |    | Q |    |
|   +---+     |--------------------------------|    +---+    |
|             |################################|             |
+-------------+                                +-------------+

Multiple sessions

    Session<------+                           +------>Session
(ICH=1, OCH=1)    |                           |    (ICH=1, OCH=1)
                 \|/                         \|/
    Session<--> Connection <---------> Connection <-->Session
(ICH=2, OCH=3)   /|\                         /|\   (ICH=3, OCH=2)
                  |                           |
    Session<------+                           +------>Session
(ICH=3, OCH=2)                                     (ICH=2, OCH=3)

        Key: ICH -> Input Channel, OCH -> Output Channel 
      

For the original diagram and the specification, you can refer 2.1.2 Communication Endpoints

The point is that the physical connection is only one, however virtual connection/session can be multiple simultaneously. However, the current implementation of AMQP on Sagittarius can't handle this due to the fact that the socket is shared by the sessions (or underlying receivers).

So, if I want to do it properly, the design must be changed sort of like this:

      +------------------+
      |    Connection    |  +--> Session1
<IN>  |  +------------+  |  |
======+==+=> socket  -+--+--+--> Session2
      |  +------------+  |  |
      |     |     /|\    |  +--> Session3
      +-----+------+-----+
            |      |
           \|/     |
       +---------------+
       |  Frame reader |
       +---------------+

And possibly, the reading frame process is running on a background thread, once the connection is established.

Let's see what I can do...

Markdown support

On Sagittarius, I was, well still am, using scribble, which the parser is ported from Chibi Scheme, to write the document. Recently, though it's been already a couple of years, I've been thinking that using one of the documentation formats might be better so that users can read better on the repository if they want to use the latest version.

So, I've updated (text markdown) library to support better Markdown. The library is based on commonmark-java, and more or less compliant with Commonmark specifications. The library keeps the old behaviour, which generates a sort of old intermediate format and HTML, though that should be considered deprecated.

The basic usage of the library can be like this:

(import (rnrs)
        (text markdown)
        (text sxml serializer))

(define (main args)
  (let ((file (cadr args)))
    (call-with-input-file file
      (lambda (in)
        (let ((node (parse-markdown markdown-parser in)))
          (srl:sxml->html-noindent
           (markdown-converter:convert default-markdown-converter 'html node)
           (current-output-port)))))))

So parse-markdown procedure parses the input textual port according to the given markdown-parser, which is one of the default parsers provided by the library. The other default parser is commonmark-parser, which strictly conforms Commonmark specification.

markdown-parser

This supports most of the GFM, footnotes and definition list

commonmark-parser

This only supports things listed on Commonmark specification. So, no table, no strikethrough or others.

The above script is actually used to generate this post. This means, obviously, the markdown-parser is used (as the code already shows :D)

NOTE: Below is the convenient (also for my memo) command to generate an HTML post.

# For Mac
sash blogpost-markdown.scm post.md | pbcopy

There's also a library called (text markdown extensions) and (text markdown converters). These libraries provide an API / framework to provide custom extensions, such as GFM, which is mostly supported by the library.

Next step

Sagittarius document requires a bit more tweaks. Below are the requirements

• Table of contents
• Index table
• eval expression
• Page split per section
• Proper anchor per pages
• Navigation bar

Most of the things are done already, just a bit more. I think after this is done, I can finally release 0.9.9.