The Artist-CD-Track sample – version 2

This is an updated version of the Artist-CD-Track sample previously posted.

  1. The syntax for defining a relvar has changed, so that only the heading need be defined as attribute names and types. The type can be inferred from a literal or variable of that type.
  2. The output format has been improved. Note the use of fold(&,) to format lines of output text.
  3. The code has been annotated to assist in understanding.
// translation of sample application from http://search.cpan.org/dist/DBIx-Class/lib/DBIx/Class/Manual/Example.pod
#noisy 0

// create the database relvars
artist  := {{ artistid:0, name:'' }}
cd      := {{ cdid:0, artistid:0, title:'', year:0 }}
track   := {{ trackid:0, cdid:0, title:'' }}

// some data in temporary relvars, with auto-generated ordinals
artist_data := {
  { name := 'Michael Jackson'}, 
  { name := 'Eminem' }
} [{ *artistid := ord() }]

cd_data := {
  { title := 'Thriller', name := 'Michael Jackson' },
  { title := 'Bad', name := 'Michael Jackson' },
  { title := 'The Marshall Mathers LP', name := 'Eminem' }
} [{ *cdid := ord() }]

track_data := {
   { title := 'Beat It'         , cd := 'Thriller' },
   { title := 'Billie Jean'     , cd := 'Thriller' },
   { title := 'Dirty Diana'     , cd := 'Bad' },
   { title := 'Smooth Criminal' , cd := 'Bad' },
   { title := 'Leave Me Alone'  , cd := 'Bad' },
   { title := 'Stan'            , cd := 'The Marshall Mathers LP' },
   { title := 'The Way I Am'    , cd := 'The Marshall Mathers LP' }
 } [{ *trackid := ord() }]

 // update the database relvars
artist := union artist_data
cd := union (cd_data join artist)[{ title, cdid, artistid,year:=0}]
track := union (track_data join cd[{ *cd := title }]) [{ trackid, title, cdid }]

// functions to answer various queries
get_tracks_by_cd(t) => cd[ title = t {*title} ] join track
get_tracks_by_artist(a) => (artist[ name = a {*name}] join cd) [{cdid}] join track
get_cd_by_track(t) => track [ title = t {cdid} ] join cd
get_cds_by_artist(a) => artist [name = a {artistid} ] join cd
get_artist_by_track(t) => (track [title = t { cdid }] join cd) [{artistid}] join artist
get_artist_by_cd(t) => (cd [title = t { cdid }] join cd) [{artistid}] join artist

// first show the raw data

crlf := h'd a'
out := crlf & "=== Sample data ===" & crlf
out := artist
out := crlf
out := cd
out := crlf
out := track

// now do the queries

show(title, data:{{str:''}}) => do {
    out := title & crlf & data[ {fold(&, "  " & str & crlf)} ]
}

out := crlf & "=== Query results ===" & crlf
show("Track title:", get_tracks_by_cd('Bad') [ {str:=title} ])
show("Track title:", get_tracks_by_artist('Michael Jackson') [ {str:=title} ])
show("CD title:", get_cd_by_track('Stan') [ {str:=title} ])
show("CD title:", get_cds_by_artist('Michael Jackson') [ {str:=title} ])
show("Artist:", get_artist_by_track('Dirty Diana') [ {str:=name} ])
show("Artist:", get_artist_by_cd('The Marshall Mathers LP') [ {str:=name} ])

The output now looks like this.

 == Sample data ===

artistid | name
--------------------------
       0 | Michael Jackson
       1 | Eminem


cdid     | artistid | title                   | year
--------------------------------------------------------
       0 |        0 | Thriller                |        0
       1 |        0 | Bad                     |        0
       2 |        1 | The Marshall Mathers LP |        0


trackid  | cdid     | title
-------------------------------------
       0 |        0 | Beat It
       1 |        0 | Billie Jean
       2 |        1 | Dirty Diana
       3 |        1 | Smooth Criminal
       4 |        1 | Leave Me Alone
       5 |        2 | Stan
       6 |        2 | The Way I Am

=== Query results ===

Track title:
  Dirty Diana
  Smooth Criminal
  Leave Me Alone

Track title:
  Beat It
  Billie Jean
  Dirty Diana
  Smooth Criminal
  Leave Me Alone

CD title:
  The Marshall Mathers LP

CD title:
  Thriller
  Bad

Artist:
  Michael Jackson

Artist:
  Eminem

Leave a Comment

Filed under Code sample

The Artist-CD-Track sample

The original Artist-CD-Track sample can be found here:  http://search.cpan.org/dist/DBIx-Class/lib/DBIx/Class/Manual/Example.pod.

The Muldis-D translation can be found here: http://muldis.com/CD.html.

The Andl implementation is below. Note that they are not strictly comparable because:

  1. Andl provides no persistence. The results are only in memory.
  2. Andl provides limited output formatting.
artist  := {{ artistid:=0, name:='' }} [false]
cd      := {{ cdid:=0, artistid:=0, title:='', year:=0 }} [false]
track   := {{ trackid:=0, cdid:= 0, title:='' }} [false]

artist_data := {
  { name := 'Michael Jackson'}, 
  { name := 'Eminem' }
} [{ *artistid := ord() }]
artist := artist union artist_data

cd_data := {
  { title := 'Thriller',                name := 'Michael Jackson' },
  { title := 'Bad',                     name := 'Michael Jackson' },
  { title := 'The Marshall Mathers LP', name := 'Eminem' }
} [{ *cdid := ord() }]
cd := cd union (cd_data join artist)[{ title, cdid, artistid, year:=0}]

track_data := {
   { title := 'Beat It'         , cd := 'Thriller' },
   { title := 'Billie Jean'     , cd := 'Thriller' },
   { title := 'Dirty Diana'     , cd := 'Bad' },
   { title := 'Smooth Criminal' , cd := 'Bad' },
   { title := 'Leave Me Alone'  , cd := 'Bad' },
   { title := 'Stan'            , cd := 'The Marshall Mathers LP' },
   { title := 'The Way I Am'    , cd := 'The Marshall Mathers LP' }
 } [{ *trackid := ord() }]
track := track union (track_data join cd[{ *cd := title }]) [{ trackid, title, cdid }]

get_tracks_by_cd(t) => cd[ title = t {*title} ] join track
get_tracks_by_artist(a) => (artist[ name = a {*name}] join cd) [{cdid}] join track
get_cd_by_track(t) => track [ title = t {cdid} ] join cd
get_cds_by_artist(a) => artist [name = a {artistid} ] join cd
get_artist_by_track(t) => (track [title = t { cdid }] join cd) [{artistid}] join artist
get_artist_by_cd(t) => (cd [title = t { cdid }] join cd) [{artistid}] join artist

get_tracks_by_cd('Bad') [{ i'Track title' := title }]
get_tracks_by_artist('Michael Jackson') [{ i'Track title' := title }]
get_cd_by_track('Stan') [{ i'CD title' := title }]
get_cds_by_artist('Michael Jackson') [{ i'CD title' := title }]
get_artist_by_track('Dirty Diana') [{ i'Artist name' := name }]
get_artist_by_cd('The Marshall Mathers LP') [{ i'Artist name' := name }]

The output looks like this.

Track title
---------------
Dirty Diana
Smooth Criminal
Leave Me Alone

Track title
---------------
Beat It
Billie Jean
Dirty Diana
Smooth Criminal
Leave Me Alone

CD title
-----------------------
The Marshall Mathers LP

CD title
----------
Thriller
Bad

Artist name
---------------
Michael Jackson

Artist name
-----------
Eminem

Leave a Comment

Filed under Code sample

Transitive closure

I figured out how to write the code for Transitive Closure. Here it is in Andl.

// define recursive function
XYT := {{X:='',Y:=''}}
TRANCLO:XYT(XY:XYT) => do {
  TTT := XY[{*Z := Y}] compose XY[{*Z := X}] union XY if(TTT = XY, TTT, TRANCLO(TTT))
}
// call it
TRANCLO(MM[ {X:=MAJOR_P#, Y:= MINOR_P# } ]) [{MAJOR_P#:=X, MINOR_P#:=Y }]

Note

  1. XYT is an expression which provides the type information for what amounts to a function call. This is structural typing, there are no type names.
  2. The symbol ‘=>’ means deferred evaluation, in this case with arguments. A function call by any other name.
  3. The do {} block allows an arbitrary number of statements, and returns the value of the last expression.

I have to confess I’m quite pleased with the outcome.  Here is the equivalent code in Tutorial-D.

/* the function */
OPERATOR TRANCLO ( XY RELATION { X P#, Y P# } ) RETURNS RELATION { X P#, Y P# } ;
    RETURN 
  ( WITH ( XY UNION ( ( XY RENAME ( Y AS Z ) ) 
              COMPOSE ( XY RENAME ( X AS Z ) ) ) ) AS TTT :
    IF TTT = XY THEN TTT /* unwind recursion */
      ELSE TRANCLO ( TTT ) /* recursive invocation */
    END IF 
  ) ;
END OPERATOR ;
/* the call */
( TRANCLO ( MM RENAME ( MAJOR_P# AS X , MINOR_P# AS Y ) ) ) RENAME ( X AS MAJOR_P# , Y AS MINOR_P# )

The similarities are reasonably obvious.

Leave a Comment

Filed under Code sample

Site under construction

Patience is a virtue.

Site under construction.

Older posts have been reconstructed from posts to the TTM mailing list, with some minor edits.

Leave a Comment

Filed under General

The Philosophy of Andl

The philosophy of the Andl language includes the following considerations.

  1. Consistent use of syntax. Things that do similar things should look similar, and vice versa.
  2. Functional in style (not procedural or OO). A language of expressions with (almost) no side-effects, which can support either lazy or eager evaluation. Everything that looks like a value could actually be an expression.
  3. Lexical scoping. Predefined->catalog(s)->argument(s)->current tuple(s)->local. No scope resolution operators, just renaming and aliases. Scoping reads left-to-right, consistent with infix notation.
  4. Few or no reserved words. System library functions have names, but can always be overridden by user definitions.
  5. Not too many arcane symbols. They just take too much explaining.
  6. Lexical: any legal characters are permitted in an identifier and any legal characters can be used to construct a literal character string (but control characters are not recognised). Strings concatenate. Special quoting conventions for identifiers, Unicode strings, time literals.
  7. Types: the native abstract types are logical, number, character, time and binary. All SQL types are accepted, but are converted accordingly.
  8. The compiler is hand-coded LL(1+), recursive descent, which limits certain kinds of syntax. Shouldn’t be a problem in practice. The grammar is quite small.

Leave a Comment

Filed under Uncategorised

Image relations

TTM and its companion book DTATRM talk about an IMAGE RELATION function, that joins two relations, embedding matching tuples from the right as a relation-valued attribute of the left.

Here is a piece of code that creates an RVA. The relation exam_mark is partitioned across the several courses.

cer := course [{ CourseId, ExamResult := {{ * }} joinr exam_mark}]

CourseId   | ExamResult
----------------------------------
C1         | StudentId  | Mark
           | ---------------------
           | S1         |       85
           | S2         |       49
           | S4         |       93
C2         | StudentId  | Mark
           | ---------------------
           | S1         |       49
C3         | StudentId  | Mark
           | ---------------------
           | S3         |       66
C4         | StudentId  | Mark
           | ---------------------

Here is a piece of code that does the job of putting the tuples back together again.

cer [{ fold(union,ExamResult) }]

StudentId  | Mark
---------------------
S1         |       85
S2         |       49
S4         |       93
S1         |       49
S3         |       66

Note that this relies on an anonymous attribute, which is ‘lifted’ out as a single value.

Leave a Comment

Filed under Language

99 bottles of Andl beer!

In response to a challenge, I decided to actual solve the problem. Here it is. The lyrics are here: http://www.99-bottles-of-beer.net/lyrics.html

ten := {{ n:=0 },{ n:=1 },{ n:=2 },{ n:=3 },{ n:=4 },{ n:=5 },{ n:=6 },{ n:=7 },{ n:=8 },{ n:=9 }}
hundred := (ten join ten[{nn:=n}]) [ {neg := -10*nn-n,nnn := 10*nn+n}] [nnn>0]
line1 := hundred [{ neg, text := nnn || " bottle" || if(nnn=1,"","s") || " of beer on the wall, " || 
         nnn || " bottle" || if(nnn=1,"","s") || " of beer."}]
line2 := hundred [{ neg:=neg+0.5, text := "Take one down and pass it around, " || (nnn-1) || " bottle" ||
         if(nnn=2,"","s") || " of beer on the wall."}]
line3 := "No more bottles of beer on the wall, no more bottles of beer."
line4 := "Go to the store and buy some more, 99 bottles of beer on the wall."
(line1 union line2) [$] [{text}] union {{text:=line3},{text:=line4}}

The first two lines just generate the sequence. Later there will be a better way to do that. The 4 lines of text are dictated by the song lyrics. Most of the program is fiddly details to precisely match the song lyrics as published, and get the lines in the right order. The end result is a relation containing the song lyrics in the correct order, which is then simply printed.

I found this quite a fascinating exercise. I had to implement IF(,,) but I think the only other control structure needed is user-defined recursive functions. The result is a programming paradigm quite unlike any that I’m familiar with, and surprisingly powerful. And weird.

Leave a Comment

Filed under Code sample, Language

6 questions to answer

These are questions to be resolved because my vision for Andl depends on them.  The questions are: how to build on the work of TTM and TD to better address the ability to do:

  1. Complex aggregation (including things like
    1. statistical variance
    2. running sum
    3. aggregations across ‘outer joins’.
  2. Deep ‘self joins’, that is relations that form graphs and trees (and including aggregation on them, such as pricing a bill of materials).
  3. Deep nesting, that is relations that are not in first normal form, with RVAs (relation valued attributes) that in turn include RVAs to form trees and graphs in a single relation.
  4. Relational (joinable) functions.
  5. Relational Updates (not so SQL-like).
  6. Quantifiable restriction (equivalents for SQL TOP/LIMIT; list parts obtainable from at least 2 suppliers)

I think the ground work has been laid already, and all I need to do is to draw it together.

Leave a Comment

Filed under Rationale

Out of the tar pit

A really good paper. Definitely worth a read. See here: http://shaffner.us/cs/papers/tarpit.pdf. Ben Moseley and Peter Marks.

Complexity is the single major difficulty in the successful development of large-scale software systems. Following Brooks we distinguish accidental from essential difficulty, but disagree with his premise that most complexity remaining in contemporary systems is essential. We identify common causes of complexity and discuss general approaches which can be taken to eliminate them where they are accidental in nature. To make things more concrete we then give an outline for a potential complexity-minimizing approach based on functional programming and Codd’s relational model of data.

There is nothing in the paper that should have surprised a relational audience. Indeed they might have sat back and quietly nodded or applauded at the appropriate places. Mind you, it’s pretty basic stuff and lacks higher order operations.

There was quite a bit to puzzle or even offend a TTM advocate, in the comments about a suitable language. The type system, the structure and even the scope and purpose of the language would be hard to reconcile with D. Some of these are mentioned on p63.

I saw a Feeder as a way to obtain data from a non-relational source, which will necessarily result in the execution of an INSERT/UPDATE/DELETE operation. An Observer would be a way for external logic to execute as a consequence of a change in relational state (the issue of trigger vs polling is unimportant). The result could be as simple as updating a screen display or something more complex like sending an email or synchronising with another system. These are very MVC-like concepts.

My main disappointment with the paper is that the (hoped for) final section is missing or severely truncated. The expose of problems is excellent as far as it goes, but the fragments of concrete solution presented are unsatisfying. This paper got some attention here: http://lambda-the-ultimate.org/node/1446. Moseley released some source code, but does not seem to have worked on this much since about 2006. See: https://groups.google.com/forum/?fromgroups#!topic/frp-discuss/BNmBgtqRUFY.

In a nutshell, he captures what I would like to do, but doesn’t help all that much with solving the problem of how to do it.

Leave a Comment

Filed under Rationale

The true calling of ‘D’

Much of TTM and related writings deals with what’s wrong with SQL and how it should be done better. SQL is an essential part of the communications between applications and databases, in conjunction with an ORM of some kind. Which kind of points to a similar role for the language D. At least that’s the way it has seemed to me.

My question is: would it be better to think of D not so much as a replacement for SQL as the language in which to code an application data model?

Using a slightly modified version of my 4 layers, and just thinking about a modest web app:

  1. UI access: coded in HTML, CSS and JS.
  2. Glue code: written in GP language: Java, C#, ruby, etc
  3. Data model: coded in GP language.
  4. DBMS access: coded in GP language and SQL.

By data model here I mean the totality of the state of the business data that models the application, both transient and persistent. The idea would be to code layer 3 entirely in a suitable D. It would draw together data from a variety of sources, and is free to use SQL and a DBMS for persisting or retrieving data.

And that leaves me pondering two questions.

  1. What specific D features are required to fully implement a data model? Scoped constructs like functions or modules are vital, I think.
  2. What should the API between application and data model look like? POCOs rather than relvars I think.

All of a sudden this sounds like a bigger project.

Leave a Comment

Filed under Rationale, TTM