Testing your lua scripts for nginx is easy

October 12, 2013

A friend and I were extending Nginx by using  lua scripts on it. Just in case you don’t know, to enable lua scripting at nginx you can use a lua module you can read more about how to push Nginx to its limits with Lua.

Anyway we were in a cycle:

  • We did some lua coding.
  • Restart nginx.
  • Test it manually.

And this cycle was repeating until we have what we want. This was time consuming and pretty boring as well.

We then thought we could try to do some test unit with the scripts. And it was amazingly simple. We created a file called tests.lua and then we import the code we were using on nginx config.

package.path = package.path .. ";puppet/modules/nginx/functions.lua.erb"
require("functions")

We also created a simple assertion handler which outputs function name when it fails or pass.

function should(assertive)
local test_name = debug.getinfo(2, "n").name
assert(assertive, test_name .. " FAILED!")
print(test_name .. " OK!")
end

Then we could create test suit to run.

function it_sorts_hls_playlist_by_bitrate()
local unsorted_playlist = [[#EXTM3U
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1277952
    stream_1248/playlist.m3u8
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=356352
    stream_348/playlist.m3u8
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=485376
    stream_474/playlist.m3u8]]

local expected_sorted = [[#EXTM3U
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=356352
    stream_348/playlist.m3u8
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=485376
    stream_474/playlist.m3u8
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1277952
    stream_1248/playlist.m3u8]]

should(sort_bitrates(unsorted_playlist) == expected_sorted)
end

I think that helped us speeding up a lot our cycle. Once again, by isolating a component and testing it, it’s a great way to make us productive.


Testing in python can be fun too!

May 29, 2013

Humble beginning

I come from (lately) Ruby,  Java and Clojure and currently I’m working with some python projects and I was missing the way I used to test my code with rspec.  After a quick research, I found three great projects that helps to make more readable tests, they are: py.test, Mock and sure.

Better assertions

I was missing a better way to make asserts into my tests. The option about using the plain assert is good but not enough and by using sure I could do some awesome code looking similar to rspec.

#instead of plain old assert
assert add(1, 4) == 5
#I'm empowered by
add(1, 4).should.be.equals(5)
[].should.be.empty
['chip8', 'schip8', 'chip16'].shouldnt.be.empty
[1, 2, 3, 4].should.have.length_of(4)

And this makes all the difference, my test code now is more expressive.

Struggling with monkeypatch

The other challenge I was facing was to understand and use monkeypatch for mocks and stubs.  I find easier to use the Mock library even though its @patch looks similar to monkeypatch but I could grasp it quickly.

#Stubing
def test_simple_math_remotely_stubed():
  server = Mock()
  server.computes_add = Mock(return_value=3)

  add_remotely(1, 2, server).should.be.equals(3)

#Mocking
def test_simple_math_remotely_mocked():
  server = Mock()

  add_remotely(1, 2, server)

  server.computes_add.assert_called_once_with(1, 2)

#Stubing an internal dependency
@patch('cmath.nasa.random')
def test_simple_math_with_randon_generated_by_nasa(nasa_random_generator):
  nasa_random_generator.configure_mock(return_value=42)

  add_and_sum_with_rnd(3, 9).should.be.equals(54)

#Mocking an internal dependency
@patch('cmath.mailer.send')
def test_simple_math_that_sends_email(mailer_mock):
  add_and_sends_email(3, 9)

  mailer_mock.assert_called_once_with(
          to='master@math.com',
          subject='Complex addition',
          body='The result was 12')

Make sure you

  1. Are using virtualenv for better lib version managment
  2. Have installed pytest, sure and mock
  3. Git cloned the project above to understand it.

Redundancy and failover on your life

May 19, 2013

Very simplified introduction: failover is the ability to keep using a service or device in case it fails, and you usually achieve that by having redundancy, having more than one service or device at time. A silly example could be when the power goes off in your house, you can handle that (failover) by having and using a flashlight (redundancy) as backup light system.

In my life I faced similar problems all the time and I think it is valid to share that. I’ll start with the very basic service (but currently very necessary) Internet, suppose we’re not at home or we’re travelling or our beloved ISP is off, I deal with that by having an extra 3G modem and Kindle with 3G free-worldwide.

travel quite often and everytime sometimes I face issues with outlets.  My notebook is meant to be used on Brazil outlet pattern but when I go to US I need to use an outlet adapter. It is not a very accurate failover mechanism however travelling with one world outlet adapter can save you from some pain.

The main way I have fun is by playing games; then in case of my console broke or the power went out I have the portable, again it’s not quite accurate a failover mechanism but for my purpose it is.

Another area is TV, suppose my TV was stealed I can keep watching it by using my usb tv or even my gps tv.

Going to digital world I can tell you endless stories and ways to have failover. The most obvious could be have your files in your computer but keep it also in a cloud storage. I love this thing about digital buying, I used to buy games digitally (Steam, eShop, PSN) and even though I change computer I don’t need dirty old DVD’S to recovery my games, they are all associated with my account.

These last two are the best IMHO: make all the documents you have digital copy (this is easy today since any smartphone can take pictures), try to attached them to cloud (email, storage…), it saved me a lot of time. You should have at least two phone numbers of a service (food delivery, cab, hospital and etc.) because sometimes you don’t have easy access to get this info.

And you, what do you do to have failover in your life?


Clojure session #01

June 17, 2012

Since I’m not posting nothing here lately, I’ve decided to write some thoughts and examples with some subject, this time is Clojure.

Multiple arities

(defn sum
"sum given numbers"
([] 0)
([x] (+ x))
([x y] (+ x y)))

Undefined number of arguments

(defn sum [& all] (reduce + all))

Local bindings

This can help you to bind some values and let your code more readable

(let [x 2] (+ x 2)) ; x only exists within let context

doing sides effects (maybe you can see this like blocks)

(do
(println "hey")
(def a (read))
(println " you typed " a))

Closures in clojure

(defn validnumber? [max] #(if (> % max) false true))
; or (defn validnumber? [max] (fn [number] (if (> number max) false true)))

(def bellow100? (validnumber? 100))

(bellow100? 9) ; true
(bellow100? 200) ; false

Curying

(def plus-one (partial + 1))
(plus-one 2)

Compose functions

(def composed (comp #(+ 2 %) +))
(composed 1 2)

Thrush operators ( -> and ->> )

-> data flow (in front) among functions
->> data flow (after) among functions

(-> 4 (- 2) (- 2))
;translates to
(- 4 2) (- 2) => (- 2 2) => 0

(->> 4 (- 2) (- 2))
;translates to
(- 2 4) (- 2) => (- 2 -2) => 4

another good usage for ->

(def person {:name "name" :age 120 :location { :country "US" } })
;instead of (:country (:location person)) a more readable way could be
(-> person :location :country)

PS: hardly based on good book Pratical Clojure


Will we only create and use dynamic languages in the future?

January 7, 2012

Since I’ve playing with some dynamic languages (Ruby and Clojure), I have been thinking about why would anybody create a new static typed language?! And I didn’t get the answer.

I started programming in Visual Basic and I taste its roots, which are almost all full of procedure commands (bunch of do, goto and end), then I moved to C#, sharper it changes the end’s for }’s and give us a little more power based on some premises: we can treat two different things in the same way, polymorphism. The last static language, but not the least, I used (and I use it) Java, abusing of his new way of treating a set of things equality, the interfaces and using its “powers” on reflections.

Although when I started to use Ruby I saw that I could treat a group of things equality without doing any extra work. I still need to code models and composed types, even though we can create or change them dynamically using “real power” of metaprogramming.

When I start to study and apply the Clojure and its principles, my first reaction was the rejection, how can I go on without my formal objects, how can I design software without a model in the head and so on. I wasn’t thinking about how actually I do software, currently I use TDD to design software and I don’t think what models I need to have, I do think in terms of “what I want”. At minimum, Clojure make me think about, do we really need object to design software?! .  A three days ago I saw an amazing video about similar thoughts: Some thoughts on Ruby after 18 months of Clojure.

Summarising: With my limited knowledge of theses languages, let’s suppose we use a function (which we don’t have source code) and we want to do something before that function is executed (intercept) using: VB I’ll need to check every single piece of code which we call this function and call another one, in Java we can use a AOP framework, in Ruby we can use the spells of metaprogramming. It seems that some frameworks, patterns and extra work aren’t needed more because of this dynamic language evolution.

My conclusions using dynamic languages (Clojure/Ruby) for now it’s: I write less code and reuse them more easy, so I don’t see any reason to create/use a new static typed language, would you see any motivation to do that?

PS: When I use C# (.Net Framework 1.3 – 2.0) it was not so super cool as today.


Clojure resources

November 29, 2011

Always that I start to learn a new language, I promise to keep the best resources links I found, but it never works. This post suppose to be updated often. Any broken link or suggestion, just comment and I’ll try to fix, add or remove it.

Links, tutorials, guides, documentations, screencasts and etc.

  1. Clojure official site
  2. Installing Clojure, clojure-contrib and setup EMACS.
  3. Vim and “Slime”
  4. VIM for Clojure
  5. Quick-start with examples
  6. VIDEO – Great short introduction videos for Clojure!
  7. VIDEO – Introduction to logic programming with Clojure
  8. VIDEO – Clojure for Java Programmers 1 of 2
  9. VIDEO – Functional programming by UCBerkeley 
  10. VIDEO – Great tutorial for Clojure focused on concurrency
  11. Try code clojure online
  12. Leiningen tutorial for beginners
  13. Midje – A test framework for Clojure
  14. Clojars – Community repository for open source clojure libraries

Books


Functional programming with Clojure

November 27, 2011

Clojure

I’ve been studying the new language called Clojure (all the cool kids are talking about Clojure). It is a functional language created by Rich Hickey around 2007. This is a(nother) dialect of Lisp. It is a dynamic language as Ruby, JavaScript and others. As said before Clojure (pronounced as closure) it’s a impure functional language in contrast with Haskell, a pure functional language. It runs over the JVM, so it’s fast, interoperable with Java among a lots of good stuffs that JVM give us. To put hands-on and try code something you can use the try Clojure online or you can download the clojure.jar file and run it. Surprisingly Clojure it’s easy to learn.


java -jar clojure-x.x.x.jar

What it a functional language? (concepts)

first-order functions -> functions are treated as values. You can store a function on a variable, you can pass one function to another or you can return a function from another function.

var sum = function(a,b){
  return a + b;
};

var obj = function(sum){
  return {
    hello: "hello",
    sum: sum
  };
}();

obj.sum(3,5);

functions constructs -> the language constructs are function instead of keyword. Constructions for conditions (if), for iterations (for, while), catch exceptions (try, catch) and others.


(if condition do-it else-do-it)

stateless -> it’s functional in the sense of math, you have functions which defines values input and output and doesn’t rely on outside global state. In such pure function you won’t produce any side-effect (read, write outside resource). Obviously we will produce programs which causes side-effects, clojure helps you build “mutable” data . On other pure languages like Haskell side-effects are treated as expections so you have concepts like actors and monad.

immutable data -> collections and local variable, in clojure, are immutable. The immutability, helps us in parallelism, since the “values” are immutable you can shared then without worry about locks.

currying -> is the technique of transforming a function that takes multiple arguments (or an n-tuple of arguments) in such a way that it can be called as a chain of functions each with a single argument (partial application).

memoization -> is an optimization technique used primarily to speed up computer programs by having function calls avoid repeating the calculation of results for previously processed inputs.

Resources


Follow

Get every new post delivered to your Inbox.