The Anagrambler

After running a few more tests we eventually convince ourselves that NN.cicada is working, so we open a new file in our Cicada directory and start thinking about how to put our networks to use.

The particular learning algorithm we are using is well suited to the task of pattern completion. We will demonstrate by building a network to unscramble anagrams. The inputs to this network will be the number of times each of the 26 letters appears in a word, encoded in the activity levels of 26 input neurons. The outputs will be the ordering of those letters relative to alphabetical, using n output neurons for a maximum word length n. (For example, a lowest-to-highest ranking of outputs of 3-2-4-1-5 for the input ‘ortob’ would imply the ordering 3-2-4-1-5 of the characters ‘b-o-o-r-t’, which spells ‘robot’.)

anagrambler.cicada

    forEach :: {
       
       counter :: int
       
       code
       
       for counter in <1, top(args[1])> &
           args(args[1][counter], counter)
    }
   
   
    anagrambler :: neural_network : {
       
       setupNN :: {
           
           ltr :: string
           params :: { step_size :: learning_rate :: double }
           
           
           code
           
           params = { .5, .1 }
           if trap(
               the_word = args[1]
               (params<<args)()
           ) /= passed then (
               printl("Error: optional params are step_size, learning_rate")
               return     )
           
           forEach(NN_in;
               ltr =! alph[args[2]]
               args[1] = find(the_word, ltr, 0)   )
   
           NN_out[^size(the_word)]
           NN_out[*].letter =! the_word
       }
       
       
       ask :: setupNN : {
           
           outputString :: string
           
           
           code
           
           run(NN_in, params.step_size)
           
           sort(NN_out, 2)
           
           NN_out[^numOutputs]
           NN_out[*].order = activity[<numInputs+2, numInputs+numOutputs+1>]
           if size(the_word) < numOutputs then NN_out[^size(the_word)]
           
           sort(NN_out, 1)
           outputString =! NN_out[*].letter
           
           print(outputString)
       }
       
       
       teach :: setupNN : {
           
           c1 :: int
           
           
           code
           
           forEach(NN_out; args[1].order = args[2]/size(the_word))
           sort(NN_out, 2)
           
           NN_out[^numOutputs]
           
           for c1 in <1, args[2]> &
               run(NN_in, NN_out[*].order, params.step_size, params.learning_rate)
       }
    }
   
   
    the_word :: string
    NN_in :: [26] double
    NN_out :: [anagrambler.numOutputs] { order :: double, letter :: char }
   
    alph := "abcdefghijklmnopqrstuvwxyz"
   

At last, we’re ready to build a digital brain and put it to the task of unscrambling anagrams. We run Cicada, then load each of the two .cicada source files.

    > run("NN")
   
   
    > run("anagrambler")
   

Next we specify how big of a brain we need. Let’s decide to work with words of 6 or fewer characters (so, 6 output neurons), and of course we expect a 26-character alphabet (lowercase only please). So we enter the following line at the command prompt.

    > anagrambler.init(26, 6, 0)
   

With the custom brain built and ready, we can try

    > anagrambler.ask("lleoh")
   
    ehllo
   

Hardly a surprise; we haven’t taught it its first word yet.

    > anagrambler.teach("hello", 10)   | 10 = # training cycles
   
   
    > anagrambler.ask("lleoh")
   
    hello
   

Thus concludes our Cicada demonstration. Of course we’ve barely probed our anagrambler’s intelligence, but a great virtue of the interactive command prompt is that the experimental cycles are very short. For example, the author was taught it to perfectly recall three words (hello, tomato, yazoo) within a minute of coaching. But maybe the network can learn even faster -- is 10 rounds of training on each word too many? Will our network learn faster if we increase higher learning rate, or will it become unstable? It’s simple to test.

    > anagrambler.teach("hello", 5; learning_rate = that*2)
   

We might also play around with the network architecture, for example by increasing the number of output neurons to allow the anagrambler to memorize longer words. Or we could add hidden neurons to increase the complexity of its calculations, using the third argument of init().

Hopefully this example shows the great advantage of marrying C code to an interpreted environment. We’ll conclude by explaining the general procedure for incorporating C/C++ functions into Cicada.

Last update: May 8, 2024