SBlog

Useful Slick Implicits

Fri, 15 Sep 2017 08:48:00 +0000

I have used slick successfully in many projects. It offers some type safety while still allowing a great deal of control over the executed SQL code.

This control comes at a price, a big portion of slick code is repetitive and it often leads to APIs difficult to use and maintain.

While working on many different projects, we noticed a few common patterns that can be abstracted and reused across projects using slick.

The purpose of this post is not to say slick is better than any other orm-like framework, if you are interested in that subject read this Sofware Mill blob post.

During this blog post I assume you are fairly familiar with slick and it's APIs, so you might want to read slick documentation.

All examples here are open sourced and developed at Advanced Telematic Systems.

Example Schema

For the following examples I'll be using the following schema:

object Schema {
  case class Book(id: Int, title: String)

  class BooksTable(tag: Tag) extends Table[Book](tag, "books") {
    def id = column[Int]("id", O.PrimaryKey)
    def repoId = column[String]("title")

    override def * = (id, title) <> ((Book.apply _).tupled, Bok.unapply)
  }

  val books = TableQuery[BooksTable]
}

Handling Errors

Integrity Constraint Violations

Slick often returns JDBC exceptions when a query causes integrity constraint violations. These exceptions are cumbersome to handle and usually we would like to convert them to specific app level exceptions that can be handled by our business logic. This would mean writing code like this:

(books += book).asTry.flatMap {
  case Success(v) => DBIO.successful(v)
  case Failure(e: SQLIntegrityConstraintViolationException) =>
      DBIO.failed(BookAlreadyExistsError)
  case Failure(e) =>
      DBIO.failed(e)
}

With the help of DbioActionExtensions we can rewrite this as:

(books += book).handleIntegrityErrors(BookAlreadyExistsError)

implicit class DbioActionExtensions[T](action: DBIO[T]) {
  def handleIntegrityErrors(error: Throwable)(implicit ec: ExecutionContext): DBIO[T] = {
    action.asTry.flatMap {
      case Success(i) =>
        DBIO.successful(i)
      case Failure(e: SQLIntegrityConstraintViolationException) =>
        DBIO.failed(error)
      case Failure(e: BatchUpdateException) if e.getCause.isInstanceOf[SQLIntegrityConstraintViolationException] =>
        DBIO.failed(error)
      case Failure(e) =>
        DBIO.failed(e)
    }
  }
}

Handling unexpected query results

Handling Missing Rows

When building slick DBIO instances you often end up with DBIO[Option[T]], since the query can return an empty result set. But this is awkward to handle as you then need to pattern match on the DBIO result just to return an application level error:

val q = books.filter(_.title === "1984").result.headOption.flatMap {
  case Some(book) => DBIO.successful(book)
  case None => DBIO.failure(BookNotFound)
}

This can be re written if we have DBIOOptionOps in scope:

books.filter(_.title === "1984").result.failIfNone(BookNotFound)

implicit class DBIOOptionOps[T](io: DBIO[Option[T]]) {
  def failIfNone(t: Throwable)
                (implicit ec: ExecutionContext): DBIO[T] =
    io.flatMap(_.fold[DBIO[T]](DBIO.failed(t))(DBIO.successful))
}

Fetching Rows by Id

We often need to get a specific row of a table. You might have a primary key on the table, so you know there can only be one row for that key, but with slick there is no short way to find a row by some key. This means we need to write something like this:

books.filter(_id === id).result.headOption.flatMap {
    case Some(e) => DBIO.successful(e)
    case None => DBIO.failure(BookNotFound)
}

It would be good generalize this and be able to write:

val q = someTable.filter(_id === id).result.failIfNotSingle(BookNotFound)

This can be done with the following implicit class in scope.

implicit class DBIOSeqOps[+T](io: DBIO[Seq[T]]) {
  def failIfNotSingle(t: Throwable)(implicit ec: ExecutionContext): DBIO[T] =
    DBIOOptionOps {
      io.flatMap { result =>
        if (result.size > 1)
          DBIO.failed(Errors.TooManyElements)
        else
          DBIO.successful(result.headOption)
      }
    }.failIfNone(t)
}

This class also adds failIfEmpty which can be used to transform DBIO[Seq[T]] into a failure if the resulting Seq[T] is empty.

Updating Rows

It's common to run update statements on one column, but if the query is not written properly you might end up update more rows than you want to update, or not updating rows at all, and the query will still be successful.

books.filter(_.title === "1984").map(_.name).update("2017")

With DbioUpdateActionExtensions in scope you could write:

books.filter(_.title === "1984")
     .map(_.name)
     .update("2017")
     .handleSingleUpdateError(BookUpdateError)

    def handleSingleUpdateError(result: Throwable)(implicit ec: ExecutionContext): DBIO[Unit] = {
      action.flatMap {
        case c if c == 1 =>
          DBIO.successful(())
        case c if c == 0 =>
          DBIO.failed(result)
        case _ =>
          DBIO.failed(Errors.TooManyElements)
      }
    }

This does not prevent the app from updating the rows unless this is executed in a transaction, but it does return an error that you can investigate. It will also return an error if the row to be updated does not exist and no rows were updated.

Pagination

Pagination can be abstracted in slick using the implicits in libats:

books.paginate(offset = 10, limit = 100)

books.paginateAndSort(_.id, offset = 10, limit = 100)

Conclusion

Slick is very close to SQL, which can be very powerful and very repetitive at the same time. But as you can see slick can simply be extended using implicits.

In a future blog post I will write about abstracting Slick Mappings that can be used to store custom types like io.circe.Json, eu.timepit.Refined or your own BookId.

All examples in this blog post are available at libats.

Transforming java Listeners into Scala Futures

Wed, 31 Dec 2014 22:18:00 +0000

When using Java libraries from Scala, if the library provides some sort of async interface you often need to instantiate a Java class that acts as listener. This is a bit cumbersome when working in Scala, you would expect the external library method call to return a Future[T] so you can use idiomatic monadic operations in your Scala code.

One pattern I use often to get around this limitation is to use Promise[T], a "future factory", to transform a Java listener into something that provides a Future[T] we can use.

For example, when using the twitter4j library and it's async API, you could be using a listener instance like this:

class TwitterClientUpdateListener(callbackObj: T) extends TwitterAdapter {
  override def updatedStatus(status: Status): Unit = {
    // do something with `status`, probably interacting with callbackObj
  }

  override def onException(te: TwitterException, method: TwitterMethod): Unit = {
    // do something with `te`, probably interacting with callbackObj
  }
}

And you also need to instantiate this class:

 def tweet(text: String): String = {
   val twitter = twitterFactory.getInstance
   twitter.addListener(new TwitterClientUpdateListener(callbackObj))
   twitter.updateStatus(text)
   // Mess around with callbackObj so we can get the side effects applied so we can return String
 }

Wouldn't it be nice if tweet returned a Future[String] instead? We could then embrace the async nature of the API we are using and whatever is calling tweet could operate on a Future[String] instead. We can easily use the a Promise[String] in this case:

class TwitterClientUpdateListener(val promise: Promise[String]) extends TwitterAdapter {
  override def updatedStatus(status: Status): Unit = {
    promise.success(status.getText)
  }

  override def onException(te: TwitterException, method: TwitterMethod): Unit = {
    promise.failure(te)
  }
}

def tweet(text: String): Future[String] = {
  val p = Promise[String]()
  val twitter = twitterFactory.getInstance
  twitter.addListener(new TwitterClientUpdateListener(p))
  twitter.updateStatus(text)
  p.future
}

tweet now returns a standard Future[String].

I see this problem when using many Java async API and I often use this small pattern to keep my Scala code more idiomatic.

Adding sugar to java interop using implicit conversions

Tue, 11 Nov 2014 21:32:00 +0000

One annoyance I have when using Java libraries from Scala is the need to explicitly create an anonymous class every time we need to pass a callback function.

For example, if an object a provides the possibility to install a callback whenever it finishes doing something, usually this is implemented using an anonymous class with code similar to this:

a.onFinish(new OnFinishHandler {
  override def f(result: Result) {
    do_something_with_result
  }
})

This is a very awkward idiom to use in a language that support anonymous functions. This syntax would be much nicer if the library was written in Scala, the library author could implement this like:

a.onFinish { result => do_something_result }

Wouldn't it be nice if we could write this anyway? Turns out it's possible if we define an implicit conversion from Result => Any to OnFinishHandler:

object FuncToFinishHandlerConversion {
  implicit def funcToFinishHandlerConversion(callback: Result => Any): OnFinishHandler =  {
    new onFinishHandler() {
      override def f(result: Result): Unit = f(result)
    }
  }
}

Now if we import FuncToFinishHandlerConversion._ in the classes where we need to use a.onFinish, we just need to use:

a.onFinish { result => do_something_with_result }

We do need to define an implicit conversion for each type, but it pays off if we are using that external library a lot.

Deploying an Akka Application

Fri, 10 Oct 2014 15:32:00 +0000

Deploying a Scala app is not simple. One solution is to build a big fat jar with all the dependencies using sbt-assembly, or just run the app with sbt in production.

If you are using Akka, there is a very good alternative to these options.

Ingredients

SBT

I am using sbt with the akka-sbt-plugin to generate all the jars I need to upload to the server.

Akka microkernel

You can use the included akka microkernel mechanism to deploy your app, which makes everything so much easier. You will just need to create a Bootable class that acts as a Main class and the microkernel will take care of the rest for you.

supervisord

Running akka in a microkernel is just like running a normal app, it runs in the foreground, so you need to use daemonize your app. If you don't want to mess with init.d/upstart/systemd scripts, I recommend you use supervisord and save yourself lots of trouble and painful debugging.

fabric and rsync

You will need to upload the files to a production server and run some commands there. I chose to use fabric because it is very lightweight, but you could use capistrano or some equivalent

Preparing your app

First you need to make your app run in an akka microkernel. This is fairly easy and you just need to create a Bootable class like this:

class RiepeteKernel extends Bootable {
  val system = ActorSystem("riepeteActorSystem")

  def startup() = {
    system.actorOf(RiepeteServer.props(), "RiepeteServer")
  }

  def shutdown() = {
    system.shutdown()
  }
}

You need to define both a startup and a shutdown method. In the startup method you just need to spin up whatever actors your app needs.

Preparing your build

Create a directory named /project and in there create a file named plugins.sbt with the following content:

addSbtPlugin("com.typesafe.akka" % "akka-sbt-plugin" % "2.3-M2")

This loads the akka-sbt-plugin into sbt.

You then need to create a build definition for your project. Create a <sbt root>/project/Build.scala with the following contents:

import akka.sbt.AkkaKernelPlugin
import akka.sbt.AkkaKernelPlugin.{Dist, distJvmOptions, outputDirectory}
import sbt.Keys._
import sbt._
import akka.sbt.AkkaKernelPlugin.distMainClass

object RiepeteKernelBuild extends Build {
  val Organization = "io.simao"
  val Version      = "0.0.1"
  val ScalaVersion = "2.11.1"

  lazy val RiepeteKernel = Project(
    id = "riepete-kernel",
    base = file("."),
    settings = defaultSettings ++ AkkaKernelPlugin.distSettings ++ Seq(
      libraryDependencies ++= Dependencies.all,
      distJvmOptions in Dist := "-Xms256M -Xmx1024M",
      outputDirectory in Dist := file("target/riepete-dist"),
      // Use the name of the Bootable class you created earlier
      distMainClass in Dist := "akka.kernel.Main io.simao.riepete.server.RiepeteKernel"
    )
  )

  lazy val buildSettings = Defaults.defaultSettings ++ Seq(
    organization := Organization,
    version      := Version,
    scalaVersion := ScalaVersion,
    crossPaths   := false,
    organizationName := "simao.io",
    organizationHomepage := Some(url("https://simao.io"))
  )

  lazy val defaultSettings = buildSettings ++ Seq(
    // compile options
    scalacOptions ++= Seq("-encoding", "UTF-8", "-deprecation", "-unchecked"),
    javacOptions  ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
  )
}

object Dependencies {
  object Versions {
    val Akka      = "2.3.3"
  }

  val all = List(
    "com.typesafe.akka" %% "akka-kernel" % Versions.Akka,
    "com.typesafe.akka" %% "akka-slf4j"  % Versions.Akka,
    "com.typesafe.akka" %% "akka-actor" % Versions.Akka,
    "io.argonaut" %% "argonaut" % "6.0.4"
  )
}

Of course you'll need to adapt the contents to match your app.

Now running sbt dist will create a riepete-dist directory in <sbt root>/target containing all the dependencies and a bash script that can be used to run the app. You can test your application runs executing bin/start.

Create a supervisord config file

You will now need to configure supervisord so it knows how to run your app.

Supervisord needs to run the java process itself to work properly, so we won't be using the bin/start script generated by sbt dist.

Create a file similar to this in your supervisord config directory, usually /etc/supervisord.d:

[program:riepete]
command=java -Xms256M -Xmx1024M -cp /opt/riepete/config:/opt/riepete/lib/* -Dakka.home=/opt/riepete akka.kernel.Main io.simao.riepete.server.RiepeteKernel
user=riepete
redirect_stderr=true
directory=/opt/riepete

You will again need to change these values to match your app name and the directory where it is deployed.

Writing a deploy script

To deploy a new version of the app we just need to rsync our target/riepete-dist directory into /opt/riepete in our production server.

from fabric.api import local, run, cd, put, env, path
from fabric.contrib.project import rsync_project

env.use_ssh_config = True

def sync_bins():
    rsync_project("/opt/riepete", "./target/riepete-dist/", delete=True,
                  exclude=[".git", "log", "config/application.conf", "config/riepete.conf"],
                  extra_opts="--exclude-from .gitignore")

def compile_project():
    local("sbt dist")

def restart():
    run('sudo supervisorctl restart riepete')

def deploy():
    target_dir = '/opt/riepete'

    with cd(target_dir):
        compile_project()
        sync_bins()

    restart()

Deploying

After installing supervisord on the production server and creating the app directory, you can deploy the app using:

fab -H <your-hostname> deploy

You can login into the production server and run supervisorctl tail -f <app name> to check the standard output.

Testing Akka Actors with Child Dependencies

Fri, 25 Jul 2014 17:32:00 +0000

While writing some Akka code I eventually got to a point where I needed to test that a child actor received a message from the actor I am testing, which I will call Manuel in this post.

We test this by injecting a TestProbe actor, provided by akka-testkit, into Manuel. We can do this injecting this probe as a dependency in the Manuel constructor.

The problem with this simple approach is that it will probably break your actor hierarchy. In my case I wanted Manuel to supervise its children, so I need to create these children in Manuel's actor context. If these children are injected in the constructor, then Manuel will not supervise them, because they are not actually its children. The actor that created Manuel would supervise them.

The easy way I found to do this is very java influenced, but it works and it is very simple and readable.

We just need to define a factory method on Manuel and then override this method in the test so that it returns a TestProbe instead of a real child. Then you can inspect the messages sent to the probe.

For example:

class Manuel extends Actor {
  import context.system
  val log = Logging(context.system, this)

  lazy val child = system.actorOf(Props[Child])

  def receive: Receive = {
    case m =>
        log.debug("Received a new message, notifying my child")
        child ! 'world
  }
}

And the test:

class ManuelTest extends TestKit(ActorSystem("testSystem")) with FunSuiteLike {
  test("the child receives a message") {
    val probe = TestProbe()

    val manuel = system.actorOf(Props(new Manuel {
      override lazy val child = probe.ref
    }))

    receiver ! 'hi

    probe.expectMsg('world)
  }
}

This way you can easily test the messages Manuel sends to its children.

If you need more to do have some more complex tests you can read more about the different ways around this problem in this post.

Emacs and rbenv

Thu, 19 Dec 2013 20:54:00 +0000

When writing ruby code, you need a tool to manage your ruby versions as well all the gem you need to have installed. In this post I'll show you some of the advantages of using rbenv if you are using emacs.

TLDR: Scroll down and copy past the elisp code into your .emacs to use rbenv. That's all you need.

The most popular tools to manage ruby versions are chruby, rvm and rbenv.

The best tools in a developer tool belt stay out of our way. rvm does exactly the opposite, it tries to do too much, handling gemsets, installing ruby versions, managing installed gems, etc.

chruby seems like it tries to do the right thing, doing one thing well, but unfortunately it doesn't integrate very well with emacs.

When using chruby, you need to invoke a chruby <ruby version> command in your shell before invoking bundler, gem, or any other command that uses ruby.

Emacs usually uses it's subprocess creation functions to run commands like flymake or when you run ruby or rspec in a *Compilation* buffer. This means that before you can execute a ruby process, you always need to call chruby <version> so that chruby can set all the required environment variables before your ruby process is executed.

This is not easy to manage since every emacs mode can have a different way of invoking shell commands. You could use chruby.el or something similar, but this means that all your emacs modes need to be aware of chruby.el, or you need tome automatic way to call the chruby.el API everytime you run a ruby subprocess.

But we don't need this complexity if the right environment variables and settings are always in your PATH when emacs starts a shell to execute some command.

Enter rbenv. If you make sure that emacs has rbenv's shims path in PATH everytime it starts a new command, then you don't need to worry if emacs selects the right ruby version before running your command. rbenv will do the right thing and setup all the required environment variables for you.

This means that all you need in emacs to run rbenv is the following two lines:

(setenv "PATH"
        (concat (getenv "HOME") "/.rbenv/shims:"
                (getenv "HOME") "/.rbenv/bin:" (getenv "PATH")))

(setq exec-path
      (cons (concat (getenv "HOME") "/.rbenv/shims")
            (cons (concat (getenv "HOME") "/.rbenv/bin") exec-path)))

This guarantees that everytime a subprocess command is used in emacs, rbenv's shims will be in your PATH and you don't have to worry about calling any other commands before invoking ruby commands from emacs. flymake and other common emacs modes will work without any extra configuration.

It's this simple, to use rbenv with emacs you just need these lines in your .emacs!

Ruby Pattern Matching

Mon, 10 Dec 2012 21:39:37 +0000

Don't you hate when and elsif??

Everyone needs change. I code in ruby most of the day, so I like to code in Scala just for the kicks.

One of the features I miss in Ruby after coding in scala is Pattern Matching.

Wouldn't it be nice to be able to do something like that in ruby instead of using ugly if's and whens?

Do you like this code?

class Dummy
    def initialize(x, y, z)
            @x, @y, @z = 1, 2, 3
    end

    def sum
            @x + @y + @z
    end
end

obj = Dummy.new(1, 2, 3)

# Later on... after the sun sets and the moon rises

if obj.responds_to?(:x) && obj.responds_to?(:y)
    puts "Do something awesome with dummy.x and dummy.y"
else if obj.kind_of?(Dummy)
    puts "This is a dummy"
end

Well, maybe it's ok, but it can get messy.

So I wrote a small ruby gem to be able to match objects against patterns. So the above example could be written as:

Matchmaker.match(obj) do
    pattern :x, :y do puts "Do something awesome with #{x} and #{y}" end
    pattern Dummy do puts "This is a dummy" end
end

Looks good hum?

It's even better when you are trying to match enums over a set of conditions:

array_obj = [1, 2, 3, 4]

if array_obj.size == 1
    puts "The single element is array_obj[0]"
elsif array_obj.size == 2
    first, second = array_obj.first(2)
    puts "first: #{first}, second: #{second}"
else
    first_element, others = array_obj.first, array_obj[1, array_obj.size]
    puts "head is #{first_element}, tail is #{others}"
end

This can be written with Matchmaker like this:

Matchmaker.match(array_obj) do
    enum :x do puts "The single element is array_obj[0]" end
    enum :x, :y do puts "first: #{x}, second: #{y} end"
    enum_cons :x, :xs do puts "head is #{x}, tail is #{xs}" end
end

I think it's neat :)

There are more example at the Github Repo.

Simulating a Poisson process with Ruby

Tue, 07 Aug 2012 13:46:44 +0000

If you ever worked with a big enough distributed system, you know that at some point you need to test how the system works with a large amount of traffic before deploying it to production.

One of the methods you can use to get more confident with your system is to simulate traffic on one end of the system and gather as much information as you can in order to understand what is happening.

Recently, when developing one of these systems, we needed to build a simulation that could test the system under a load of 1000 requests/minute.

The problem is that in order to do this, we shouldn't just generate 1000 events, sleep for a minute, and then generate 1000 more events. This behavior doesn't emulate the behavior of users in production at all and the system would probably perform in a very different way. What we need is a way to distribute those 1000 events over a minute, preferably in a way that resembles production users behavior.

One simple way to do this is model the users behavior as Poisson process. This way, the events will not be uniformly distributed over a minute, instead, they will follow a Poisson distribution over time.

We were developing this system in ruby, so I will explain how you can generate events that simulate a Poisson process using ruby and then check the results of observe to confirm they are close to what you would expect.

Generating events

I tried two methods to generate events modeled by a Poisson process.

Method 1: Sleeping between events

If a process can be modeled using a Poisson Process, the probability distribution of the waiting time until the next occurrence of an event is an exponential distribution 1.

This means that one way to model this system is to generate an event, sleep until the next event should be generated and then start over.

We can determine the time our process needs to sleep until the next event using the following code:

def sleepFor(rateParameter)
  -Math.log(1.0 - Random.rand) / rateParameter
end

Where rate parameter is the number of events that should happen in each unit of time. In our case, 1000 events per minute, that is about 17 events per second. The rate parameter is then our λ for our Poisson Process.

Jeff Presshing wrote a very good blog post that explains this rational in more detail.

So with our sleepFor function in place, to generate events in your system, we just need the following code:

require 'socket'

def sleepFor(rateParameter)
  -Math.log(1.0 - Random.rand) / rateParameter
end

def generate_event
  c = UDPSocket.new
  c.send('hello', 0, 'localhost', 6767)
  c.close
end

20000.times do
  generate_event

  sleep_for = sleepFor(17) # lambda = 17
  sleep(sleep_for)
end

Problem: Does `sleep` have enough granularity?

For a small rateParameter, this works as expected, see below to learn how you can confirm this. For a big number of events events per second (a big λ), the sleep function does not have sufficient granularity. This means that sleep does not actually sleep for enough time, so we need to use a different approach.

Method 2: Constantly check if an event should be generated

Another approach would be to not use sleep at all. Start an infinite loop and constantly check if we should generate an event in the current iteration. For example:

require 'socket'

LAMBDA = 17

def f_next_time(lambda)
  -Math.log(1 - Random.rand) / lambda
end

def generate_event
  c = UDPSocket.new
  c.send('hello', 0, 'localhost', 6767)
  c.close
end

start = Time.now.to_i
next_time = 0

2000.times do
  time = Time.now.to_f

  if time >= next_time
      generate_event
      next_time = time + f_next_time(LAMBDA)
  end
end

This solution is more CPU intensive, since it's executing an infinite loop checking if it should generate an event, but the results should be more precise, especially for a bigger lambda value.

Caveats

Get as much information about your system as much as you can

Your system is probably more complex than this, so you should measure and get as much information as possible in order to ensure that you are generating enough events and that those events are properly distributed in each second.

Time required to generate an event

In my example, the time to generate an event is actually negligible, but if your events take more time to generate than the inter-arrival time of the events, this will not work! Make your event generation is fast enough or try to find another solutions.

Observing events and saving data

To check if the generated events are following our Poisson process, we need to collect those events and group them by the second in which they happened, then we can analyze the results.

This can be done with the following code.

require 'json'
require 'socket'

def start_server port
  s = UDPSocket.new
  s.bind(nil, port)
  data = Hash.new(0)
  start = nil

  Signal.trap('SIGINT') do
    File.open('poisson.json','w') do |f|
      f.write(data.to_json)
    end

    puts data

    exit
  end

  while true
    msg, sender = s.recvfrom(256)
    start = Time.now.to_f if start.nil?
    bucket = (Time.now.to_f - start).round.to_i

    data[bucket] += 1
  end
end

start_server(6767)

Using this code, when the server is interrupted, using SIGINT (CTRL-C), the results will be saved to a json file than can then be analyzed.

Analyzing results

With the results collected using the above scripts, we can easily analyze the number of events generated in each second and check if they are distributed in time as we expect them to.

I used Python with matplotlib to process and visualize the results.

λ = 17 Events/Second

Since the results are already grouped by arrival second, we can plot the number of events that happened in each second.

We can then generate enough random samples using a Poisson distribution with λ = 17 and plot this data against our results.

If the plots are similar enough, then the events were generated following a Poisson process.

There are better and more exact ways to check this, for my problem this is just good enough. You should use whatever method you feel comfortable with.

With λ = 17 and using method 2, I got the following results:

As you can see, the standard deviation and mean are very close to the values you would expected (between parenthesis) and plotted data seems to also match our expectations.

Conclusion

If you are trying to simulate traffic into your system don't just throw traffic sequentially into it. Try to find a way to simulate production traffic. Even if you end up not using a perfect model of expected traffic, using some kind of model is always better than just generate uniformly distributed events.

If you want to try to model your production traffic using a Poisson process, use method 2 describe above if you don't mind the CPU intensive algorithm or use method 1 if your λ is small enough.

Don't forget to always check your results against or expectations.

MDUnify - Produce beautiful, unified markdown files

Sun, 05 Aug 2012 14:04:49 +0000

MDUnify produces a single unified and beautiful HTML document from a Markdown file.

I like using markdown to organize my text files, but when I need to send them to someone they are usually not very beautiful. The markdown compiler does not include any CSS when converting markdown to HTML, so you'll need to add them yourself.

I built a small tool that simply compiles your markdown files to HTML and includes beautiful default stylesheets to produce a nice HTML file. Currently, mdunify uses the stylesheets from the Blueprint CSS project, including the typography CSS styles, so the resulting output is a nicely formatted, typographically beautiful and easy to read document.

If your markdown document also includes images, MDUnify will download the images and include them inside your HTML file, so if you need to send the file to someone, you can send a single file instead of having to send multiple files.

You can download mdunify at my github repository.

Use Backbone JS without REST

Mon, 12 Sep 2011 22:00:02 +0000

Backbone.persistence is a simple adapter to use Backbone.js without using the REST persistence layer.

Backbone is great, but not only for applications that use a remote persistence mechanism, for example, the demo TODO app uses the HTML5 localStorage instead of the usual REST layer.

Sometimes you don't even need to use localStorage, for example, when you don't need to save data between two different sessions for the same user. If you want to run your unit tests using Rhino using Jenkins, you can't use things like localStorage.

That's where Backbone.persistance comes in, it overrides Backbone.Sync to persist all the data in memory instead of using a persistence mechanism.

You can download Backbone.persistance at my github repository.

SBlog

Useful Slick Implicits

Example Schema

Handling Errors

Integrity Constraint Violations

Handling unexpected query results

Handling Missing Rows

Fetching Rows by Id

Updating Rows

Pagination

Conclusion

Transforming java Listeners into Scala Futures

Adding sugar to java interop using implicit conversions

Deploying an Akka Application

Ingredients

SBT

Akka microkernel

supervisord

fabric and rsync

Preparing your app

Preparing your build

Create a supervisord config file

Writing a deploy script

Deploying

Testing Akka Actors with Child Dependencies

Emacs and rbenv

Ruby Pattern Matching

Simulating a Poisson process with Ruby

Generating events

Method 1: Sleeping between events

Problem: Does sleep have enough granularity?

Method 2: Constantly check if an event should be generated

Caveats

Get as much information about your system as much as you can

Time required to generate an event

Observing events and saving data

Analyzing results

λ = 17 Events/Second

Conclusion

MDUnify - Produce beautiful, unified markdown files

Use Backbone JS without REST

Problem: Does `sleep` have enough granularity?