Quote module contents into a string with Template Haskell

•10/01/2012 • Leave a Comment

I have a program with static configuration in module named Config. There is a bunch of constants like this:

extractCtxs'ctxSize :: Int
extractCtxs'ctxSize = 300
extractCtxs'sparse :: Bool
extractCtxs'sparse = True
selectWords'dropBorder :: Int
selectWords'dropBorder = 20000
selectWords'takeCount :: Int
selectWords'takeCount = 100

Now suppose I want to quote this module elsewhere and include it’s contents into an output file for record: “This file was produced with the following settings: <Config module contents goes here>”. There are many ways to do that the simplest being copying the whole module into a string and updating it everytime a change is made.

My solution involves Template Haskell. Turns out you can execute any IO action during compilation. All you really need to do is read a file and produce a literal string containing it. Here is the whole module:

{-# LANGUAGE TemplateHaskell #-}
module Config where
import Language.Haskell.TH
import Data.List.Split (splitOn)
-- START
-- extractCtxs'dumpSize :: Int
-- extractCtxs'dumpSize = 100000
extractCtxs'ctxSize :: Int
extractCtxs'ctxSize = 300
extractCtxs'sparse :: Bool
extractCtxs'sparse = True
selectWords'dropBorder :: Int
selectWords'dropBorder = 20000
selectWords'takeCount :: Int
selectWords'takeCount = 100
-- STOP
config' = $(do
 s <- runIO $ readFile "lib/Config.hs"
 return $ LitE $ stringL s)
config = head . splitOn "-- STOP" . head . tail . splitOn "-- START" $ config'

This gist contains the above module with syntax highlighting. Config.config contains the configuration part. Config.config’ contains the whole module code. Pretty simple, really.

Parametric raytracer engine in CUDA – demo

•04/02/2011 • Leave a Comment

I’ve finally finished my raytracer in CUDA yesterday :-) Today I added a quick little feature: screenshots. They can be further processed with mencoder into .avi files and put on Youtube.

Just like this one: http://www.youtube.com/watch?v=2xrfezVuhQk

I’ll write something more about the whole thing when I have time for it. In the meantime enjoy some screenshots:

https://github.com/Tener/cuda-course/commit/237ed8d3c1e746d617be5477dd320064fff709c5

They sit in Git repository on GitHub next to the code that generated them.

ffipkg: a neat bindings creation tool. Usage example.

•31/01/2011 • 4 Comments

The FFI Packaging Utility (ffipkg) is a surprisingly easy tool for creating bindings from C to Haskell. It works by processing header files and creating a bunch of modules with accessors to structures, bindings for calling functions and all the usual boilerplate you usually need to write when doing FFI. What it doesn’t do is to provide a nice, high level binding to the underlying C library. But with the boring part worked out we can do it ourselves in Haskell.

In this post I’ll show how to create a working binding to CURAND library. For those unfamiliar with CUDA: CURAND is a library for creating a high-quality pseudo random numbers directly on the graphic card’s GPU. It works with any recent NVidia card.

First let’s install ffipkg itself. Actually the package name is different – hsffig. What you need to do is:

$ cabal install hsffig

There shouldn’t be any problems with that.

The next step would be installing the CUDA toolkit. If you are on Linux and your distribution provides it prepackaged you can install it with your package manager. Otherwise proceed to NVidia site and download the right installer.

I used the one for Ubuntu and it worked very well. After running it gently asked for installation prefix and it simply unpacked into it. It also provides you with information on how to configure your system to make libraries and headers visible to the compilers.

Creating the binding library is dead simple. Just create and cd into directory, like this:

$ mkdir hs-curand && cd hs-curand

Then call ffipkg:

$ ffipkg -I${CUDA_INCLUDE_PATH} -L${CUDA_LINK_PATH} -lcurand curand.h

If the CUDA toolkit is installed system-wide you can omit the extra -I/-L options. Otherwise you should point ffipkg to the right directories, just like you would do with the C/C++ compiler. As an example, I put my installation into ~/localopt/cuda folder and my machine is 64 bit, so in my case I called: (Beware of ~ as it may not be expanded into $HOME)

$ffipkg -I$HOME/localopt/cuda/include -L$HOME/localopt/cuda/lib64 -lcurand curand.h

This command resulted in 86 files being created in current directory. Notable files:

  1. CURAND.cabal – defines the Cabal library suitable for “cabal install” or “cabal-dev install
  2. Setup.hs – a companion to .cabal file
  3. Makefile – not to be called directly. (ffipkg will silently overwrite whatever Makefile is already in the directory)
  4. A plenty of .hs files for each C datatype declared
  5. Single .hsc file with wrapping code

As we examine the files a fun fact becomes apparent: not only did ffipkg provide bindings to CURAND. The bindings to the rest of CUDA are present as well! With a simple command we have created bindings to entire or most of CUDA runtime and driver API along with the CURAND library.

The next step is an actual installation which should go fine:

$ cabal install

We have installed a library, but will it actually work? Let’s find out by writing an example program.

The CURAND documentation itself contains a small program for host API which we will use:

/* This program uses the host CURAND API to generate 100
 * pseudorandom floats.
 *
 * Code taken verbatim from CURAND library documentation.
 */
#include
#include
#include
#include
#define CUDA_CALL(x) do { if((x) != cudaSuccess) { \
      printf("Error at %s:%d\n",__FILE__,__LINE__);     \
      return EXIT_FAILURE;}} while(0)
#define CURAND_CALL(x) do { if((x) != CURAND_STATUS_SUCCESS) { \
      printf("Error at %s:%d\n",__FILE__,__LINE__);            \
      return EXIT_FAILURE;}} while(0)

int main(int argc, char *argv[])
{
  size_t n = 100;
  size_t i;
  curandGenerator_t gen;
  float *devData, *hostData;
  /* Allocate n floats on host */
  hostData = (float *)calloc(n, sizeof(float));
  /* Allocate n floats on device */
  CUDA_CALL(cudaMalloc((void **)&devData, n * sizeof(float)));
  /* Create pseudo-random number generator */
  CURAND_CALL(curandCreateGenerator(&gen,
                                    CURAND_RNG_PSEUDO_DEFAULT));
  /* Set seed */
  CURAND_CALL(curandSetPseudoRandomGeneratorSeed(gen, 1234ULL));
  /* Generate n floats on device */
  CURAND_CALL(curandGenerateUniform(gen, devData, n));
  /* Copy device memory to host */
  CUDA_CALL(cudaMemcpy(hostData, devData, n * sizeof(float),
                       cudaMemcpyDeviceToHost));
  /* Set seed */
  CURAND_CALL(curandSetPseudoRandomGeneratorSeed(gen, 1234ULL));
  /* Generate n floats on device */
  CURAND_CALL(curandGenerateUniform(gen, devData, n));
  /* Copy device memory to host */
  CUDA_CALL(cudaMemcpy(hostData, devData, n * sizeof(float),
                       cudaMemcpyDeviceToHost));
  /* Show result */
  for(i = 0; i < n; i++) {     printf("%1.4f ", hostData[i]);   }   
  printf("\n");   
  /* Cleanup */   
  CURAND_CALL(curandDestroyGenerator(gen));   
  CUDA_CALL(cudaFree(devData));   
  free(hostData);   
  return EXIT_SUCCESS; } 

When executed it will display 100 pseudo random numbers, created from the 1234 seed. For comparison here is Haskell version using our freshly created CURAND bindings:

 {-# LANGUAGE ScopedTypeVariables, CPP #-}
module Main where
import HS_CURAND_H
import System.Environment

type ElType = CFloat

genFun :: Ptr S_curandGenerator_st
       -> Ptr ElType
       -> CSize
       -> IO CInt
genFun = f_curandGenerateUniform

fi :: (Integral a, Num b) => a -> b
fi = fromIntegral

main :: IO ()
main = do
  args <- getArgs   let n :: (Read a, Num a) => a
      n = case args of
            [] -> 100
            (n':_) -> read n'

      nBytes = fi (sizeOf (undefined :: ElType) * n)

  -- allocate n data on host
  dataHost :: Ptr ElType <- mallocArray n

  -- allocate n data on device
  dataDevPtr <- malloc
  f_cudaMalloc dataDevPtr nBytes
  dataDev :: Ptr ElType <- peek (castPtr dataDevPtr)
  free dataDevPtr

  -- create pseudo-random number generator
  genPtrPtr <- malloc
  f_curandCreateGenerator genPtrPtr
       (fi e_CURAND_RNG_PSEUDO_DEFAULT)
  genPtr <- peek genPtrPtr
  free genPtrPtr

  -- set seed
  f_curandSetPseudoRandomGeneratorSeed genPtr 1234

  -- generate n data on device
  genFun genPtr dataDev n

  -- copy device memory to host
  f_cudaMemcpy (castPtr dataHost) (castPtr dataDev) nBytes
        (fi e_cudaMemcpyDeviceToHost)

  -- show result
  dataLst <- peekArray n dataHost
  putStr (concatMap (\x -> show x ++ "\n") dataLst)

  -- cleanup
  f_curandDestroyGenerator genPtr
  f_cudaFree (castPtr dataDev)
  free dataHost

  return ()

The code is mostly 1-to-1 mapping from C code, as it should be: both programs use the same API functions. The only nuisance in Haskell is the need for extra malloc/free pair whenever API function needs a pointer to variable. This part could (and should!) be done using alloca function in the real world code, but I’d rather have more understandable version of code here.

After compiling and running this program we will see almost the same output as from the C version. The only difference is caused by different behavior of printf for floats in Haskell and in C. The actual numbers are the same under the hood.

So ffipkg has actually created a working Haskell bindings for CUDA and CURAND without any sweat. Like I said already, it’s pretty low-level – much like the *-bindings packages on Hackage or even more. This can however serve as a building block for much more Haskell-like API afterwards.

It’s not always this good. There are a few downsides of ffipkg:

  1. It is hard to create a library ready for upload to Hackage
  2. Requires GNU make or equivalent (Makefiles)
  3. Documentation is disabled by default. Even if we enable it in Setup.hs by hand it still doesn’t provide a lot of useful information.
  4. Exploring the created library hard. My best find: ghci with :+m HS_library-name_H
  5. It fails for many packages with unhelpful error message
  6. Sometimes it requires post-processing of created package by hand to make it work

The taste of CUDA

•27/01/2011 • Leave a Comment

Recently I was working on some projects using CUDA. Perhaps I will write something more about this experience. Today just a few things to notice:

- CUDA is quite low level. If you want to test your algorithm fast, do learn some wrapping library. I tried Thrust – and it is an excellent library. I’m speaking here about runtime API: you really shouldn’t touch driver API if you don’t have to.

- There is a lot of publicity around CUDA, especially no NVidia sites. But then if you want to get some real information it can be hard to impossible.
- You cannot file a bug report if aren’t registered developer. Becoming registered developer is no small feat though: judging from application form you have to work in some company on some CUDA stuff, provide a web page describing your product (!) and some other silly stuff. I still didn’t get any reply for my application.
- The compiler (NVCC) has it’s bugs. Only recently I found two of them. One bug was nice enough to provide me with compiler error. Fair enough, perhaps a couple of XOR’s are too much to handle. Another was more severe: turns out sometimes the usage of -use_fast_math nvcc’s switch doesn’t really help. In my case the image that should be the same each frame was mostly random:

Cuda kernel bug - frame 1
Cuda kernel bug - frame 2

Even worse: the actual speed of the code was 100 times worse with -use_fast_math.

Another thing, not sure if this is bug or not: compiling template-heavy debugging code is a nightmare. Each target architecture could take 15 minutes and ~2 Gb (top) memory to compile – at least this was my experience.

There is plenty of things that are either broken or work poorly here. But on the other hand: CUDA is free as long as you have NVidia card and it really does terrific amount of work under the hood. In specific applications it can perform calculations dramatically faster than CPU.

Unfortunately it’s extremely easy to shoot your foot here – so badly you may have to reboot your machine, because the driver got stuck. In the long run I would expect some heavy movement towards higher level languages, most likely with some declarative/functional flavor. (No, C++ is not functional. Boost::lambda is a bad joke, and writing new class each time you want functor is stupid.)

I think one may came up with something like query-execution-evaluator: the programmer would specify the result value and the compiler would then device the best way to calculate it on given machine configuration and data set. All hairy stuff would then go away, since I really believe it’s too much to handle in the long run.

So far people that use CUDA seem to come from backgrounds that are used to bad code, since they need performance badly. If we want some other people to join in, we need better tools.

It’s JHC’s turn

•28/07/2010 • Leave a Comment

It’s been a while since I wrote here.

Today I sat down to try out the relatively new JHC – Haskell compiler created by John Meacham.

After some trivial steps I got it compiled & installed in ~/localopt/jhc-0.7.5/. It worked out of the box.

This compiler has a very interesting property: it compiles Haskell code to Ansi C. It can be processed afterwards by any way you want: cross-compiled by gcc or made into shared library. I found the latter to be pretty simple.

The plan is as follows:

  1. Create Haskell module and use “foreign export” in the file.
  2. Compile it with -fffi and dump C code (-dc).
  3. Use dumped code and a prepared template to compile shared library

Simple and clean.

First let’s create a simple Haskell module with single function “adder” and empty “main” function.

import Foreign
import Foreign.C

foreign export ccall "adder" adder :: CInt -> CInt -> CInt
adder x y = x + y

main :: IO ()
main = return ()

We will also need library initialization code. Let’s put it into some known directory, like ~/localopt/jhc-0.7.5/shared/library_init_template.c.

#define CAT(a,b) XCAT(a,b)
#define XCAT(a,b) a ## b
#define STR(a) XSTR(a)
#define XSTR(a) #a

extern void CAT (__stginit_, MODULE) (void);

static void library_init (void) __attribute__ ((constructor));
static void
library_init (void)
{
    static char *argv[] = { STR (MODULE) ".so", 0 }, **argv_ = argv;
    static int argc = 1;

    hs_init(&argc,&argv_);
    if (jhc_setjmp(&jhc_uncaught))
        jhc_error("Uncaught Exception");

}

static void library_exit (void) __attribute__ ((destructor));
static void
library_exit (void)
{
    hs_exit ();
}

Now we can compile the Haskell code:
jhc -fffi -fjgc adder_ffi.hs -o adder.bin -dc
The code will be written to file “adder.bin_code.c”. We will use it along with the library template above.


touch adder_lib.c
cat adder.bin_code.c >> adder_lib.c
cat ~/localopt/jhc-0.7.5/shared/library_init_template.c >> adder_lib.c

Now we can compile it to the shared library. The only thing we need to remember are two macros: -DMODULE=ADDER sets module name in our template, and -D_JHC_STANDALONE=0 disables default main in adder_lib.c. There are also a few default options as used by JHC.

gcc -DMODULE=ADDER -D_JHC_STANDALONE=0 -fPIC -shared -Wl,-soname,libadder.so.1 '-std=gnu99' -D_GNU_SOURCE '-falign-functions=4' -ffast-math -Wextra -Wall -Wno-unused-parameter -o libadder.so.1 adder_lib.c -DNDEBUG -O3 '-D_JHC_GC=_JHC_GC_JGC'

Now a simple test program:

#include

int adder(int x2,int x3);

int main()
{
    printf("%d\n", adder(3,4));
    return 0;
}

gcc -Wl,-rpath=`pwd` libadder.so.1 test_adder.c -o test_adder
./test_adder
7

It works!

We can wrap all needed commands into simple script (this is zsh syntax, but bash should be similar or even the same):

jhc-shared-lib () {
    jhc -fffi -fjgc $1.hs -o $1.bin -dc

    rm -f $1_lib.c
    touch $1_lib.c
    cat $1.bin_code.c >> $1_lib.c
    cat ~/localopt/jhc-0.7.5/shared/library_init_template.c >> $1_lib.c

    gcc -D_GNU_SOURCE -DMODULE=ADDER -D_JHC_STANDALONE=0 \
        -fPIC -shared -Wl,-soname,lib$1.so.1 \
        '-std=gnu99' '-falign-functions=4' -ffast-math \
        -Wextra -Wall -Wno-unused-parameter \
        -o lib$1.so.1 $1_lib.c -DNDEBUG -O3 '-D_JHC_GC=_JHC_GC_JGC'
}

Invoke it like this:
jhc-shared-lib adder

JHC looks like a lot of fun. Even if it doesn’t support all possible extensions to Haskell it does play really well with C compilers.

GHC 6.12.1 dynamic executables fun

•26/01/2010 • 1 Comment

Recently I started to play with the new GHC 6.12.1. One thing I wanted to do is to link executables with -dynamic flag. It works pretty well when invoked like this:

$ ghc --make -dynamic program.hs

The resulting program will have plenty of dependencies (50+ is pretty common). It will also be quite small: for example 26K instead of 681K.

However, when building cabalized program/library one would like to have everything done automatically. However cabal install doesn’t make -dynamic executables. There is a ticket filled about this on Cabal Trac. One possible workaround is to call cabal install with –ghc-options=-dynamic flag. The other is to include this flag in ghc-options field in .cabal file. This is better since we can add it selectively. Consider this example binlib:

name: dynbin
version: 0.0
license: BSD3
license-file: LICENSE
copyright: (c) Christopher Skrzetnicki
author: Christopher Skrzetnicki
maintainer: Christopher Skrzetnicki <gtener@gmail.com>
stability: experimental
synopsis: 
description:
category: Other
cabal-version: >= 1.2
build-type: Simple
 
library
  hs-source-dirs: src
  exposed-modules: Dynbin.Main
  build-depends: base == 4.*
 
executable dynbin-dyn
  executable: dynbin-dyn
  hs-source-dirs: src
  ghc-options: -dynamic
  main-is: dynamic_main.hs
 
executable dynbin-stat
  executable: dynbin-stat
  hs-source-dirs: src
  main-is: static_main.hs

Files used are pretty straightforward.

dynamic_main.hs:

import qualified Dynbin.Main
 
main :: IO ()
main = putStrLn "dynamic_main.hs" >> Dynbin.Main.main

static_main.hs:

import qualified Dynbin.Main
 
main :: IO ()
main = putStrLn "static_main.hs" >> Dynbin.Main.main

Dynbin.Main:

module Dynbin.Main ( main ) where
 
main :: IO ()
main = putStrLn "Hello from Dynbin.Main.main !!!"

After installing it works like charm:

$ dynbin-stat
static_main.hs
Hello from Dynbin.Main.main !!!
 
$ dynbin-dyn
dynamic_main.hs
Hello from Dynbin.Main.main !!!

Calling ldd confirms that first executable is static, and the second is dynamic.

$ ldd `which dynbin-stat`
    linux-gate.so.1 =>  (0xb78dd000)
    librt.so.1 => /lib/librt.so.1 (0xb78ab000)
    libutil.so.1 => /lib/libutil.so.1 (0xb78a7000)
    libdl.so.2 => /lib/libdl.so.2 (0xb78a3000)
    libgmp.so.3 => /usr/lib/libgmp.so.3 (0xb7856000)
    libm.so.6 => /lib/libm.so.6 (0xb782f000)
    libc.so.6 => /lib/libc.so.6 (0xb76e8000)
    libpthread.so.0 => /lib/libpthread.so.0 (0xb76cf000)
    /lib/ld-linux.so.2 (0xb78de000)
 
$ ldd `which dynbin-dyn` 
    linux-gate.so.1 =>  (0xb77f6000)
    libHSunix-2.4.0.0-ghc6.12.1.so => /usr/lib/ghc-6.12.1/unix-2.4.0.0/libHSunix-2.4.0.0-ghc6.12.1.so (0xb7765000)
    librt.so.1 => /lib/librt.so.1 (0xb7734000)
    libutil.so.1 => /lib/libutil.so.1 (0xb7730000)
    libdl.so.2 => /lib/libdl.so.2 (0xb772b000)
    libHSbase-4.2.0.0-ghc6.12.1.so => /usr/lib/ghc-6.12.1/base-4.2.0.0/libHSbase-4.2.0.0-ghc6.12.1.so (0xb72ae000)
    libHSinteger-gmp-0.2.0.0-ghc6.12.1.so => /usr/lib/ghc-6.12.1/integer-gmp-0.2.0.0/libHSinteger-gmp-0.2.0.0-ghc6.12.1.so (0xb729e000)
    libgmp.so.3 => /usr/lib/libgmp.so.3 (0xb7251000)
    libHSghc-prim-0.2.0.0-ghc6.12.1.so => /usr/lib/ghc-6.12.1/ghc-prim-0.2.0.0/libHSghc-prim-0.2.0.0-ghc6.12.1.so (0xb71de000)
    libHSrts-ghc6.12.1.so => /usr/lib/ghc-6.12.1/libHSrts-ghc6.12.1.so (0xb7198000)
    libm.so.6 => /lib/libm.so.6 (0xb7172000)
    libHSffi-ghc6.12.1.so => /usr/lib/ghc-6.12.1/libHSffi-ghc6.12.1.so (0xb7169000)
    libc.so.6 => /lib/libc.so.6 (0xb7022000)
    libpthread.so.0 => /lib/libpthread.so.0 (0xb7009000)
    /lib/ld-linux.so.2 (0xb77f7000)

So this is OK for now. But what happens if we choose to have some TH code? Let’s use some. The following code uses packages hslogger and hslogger-template to add some logging to our code.

{-# LANGUAGE TemplateHaskell, ForeignFunctionInterface, CPP #-}
 
module Dynbin.Main ( main ) where
 
import System.Log.Logger.TH (deriveLoggers)
import qualified System.Log.Logger as HSL
import System.Log.Logger ( updateGlobalLogger, rootLoggerName, setLevel )
 
$(deriveLoggers "HSL" [HSL.WARNING, HSL.NOTICE, HSL.INFO])
 
-- dynamic linking
#ifdef MAIN_LIB
main_lib = MAIN_LIB
foreign export ccall MAIN_LIB main :: IO ()
#endif
 
main :: IO ()
main = do updateGlobalLogger rootLoggerName (setLevel HSL.INFO)
          infoM "Oh hai!"

dynbin.cabal:

name: dynbin
version: 0.0
license: BSD3
license-file: LICENSE
copyright: (c) Christopher Skrzetnicki
author: Christopher Skrzetnicki
maintainer: Christopher Skrzetnicki <gtener@gmail.com>
stability: experimental
synopsis: 
description:
category: Other
cabal-version: >= 1.2
build-type: Simple
 
library
  hs-source-dirs: src
  exposed-modules: Dynbin.Main
  cpp-options: -DMAIN_LIB="main_lib"
  build-depends: base == 4.*, hslogger, hslogger-template
 
executable dynbin-dyn
  executable: dynbin-dyn
  hs-source-dirs: src
  ghc-options: -dynamic
  cpp-options: -DMAIN_LIB="main_lib"
  main-is: dynamic_dlopen.hs
  other-modules: Paths_dynbin
  build-depends: unix, filepath, directory
 
executable dynbin-stat
  executable: dynbin-stat
  hs-source-dirs: src
  main-is: static_main.hs

The static executable builds and works well:

$ dynbin-stat  
static_main.hs
Dynbin.Main: Oh hai!

However, dynbin-dyn doesn’t. Here is what happens:

Resolving dependencies...
Configuring dynbin-0.0...
Preprocessing library dynbin-0.0...
Preprocessing executables for dynbin-0.0...
Building dynbin-0.0...
Registering dynbin-0.0...
[1 of 2] Compiling Dynbin.Main      ( src/Dynbin/Main.hs, dist/build/dynbin-dyn/dynbin-dyn-tmp/Dynbin/Main.o )
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Loading package array-0.3.0.0 ... linking ... done.
Loading package filepath-1.1.0.3 ... linking ... done.
Loading package old-locale-1.0.0.2 ... linking ... done.
Loading package old-time-1.0.0.3 ... linking ... done.
Loading package unix-2.4.0.0 ... linking ... done.
Loading package directory-1.0.1.0 ... linking ... done.
Loading package process-1.0.1.2 ... linking ... done.
Loading package time-1.1.4 ... linking ... done.
Loading package random-1.0.0.2 ... linking ... done.
Loading package haskell98 ... linking ... done.
Loading package syb-0.1.0.2 ... linking ... done.
Loading package base-3.0.3.2 ... linking ... done.
Loading package containers-0.3.0.0 ... linking ... done.
Loading package mtl-1.1.0.2 ... linking ... done.
Loading package parsec-2.1.0.1 ... linking ... done.
Loading package network-2.2.1.7 ... linking ... done.
Loading package hslogger-1.0.7 ... linking ... done.
Loading package pretty-1.0.1.1 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package hslogger-template-1.0.0 ... linking ... done.
Loading package ffi-1.0 ... linking ... done.
src/Dynbin/Main.hs:1:0:
    Dynamic linking required, but this is a non-standard build (eg. prof).
    You need to build the program twice: once the normal way, and then
    in the desired way using -osuf to set the object file suffix.
cabal: Error: some packages failed to install:
dynbin-0.0 failed during the building phase. The exception was:
ExitFailure 1

What does it mean? Well, we can’t simply mix dynamic builds with TH. I’m not sure why is it so, but the description how to solve it when using ghc –make is here. But here we are in a cabal install world. So how we are going to solve it?

There are two solutions. The first, mentioned in discussion for ticket #600, is this: move all executables to a different package and make it depend on the first one. This solution works, but is boring ;-) Here is a .cabal file for this:

name: dynbin-prog
version: 0.0
license: BSD3
license-file: LICENSE
copyright: (c) Christopher Skrzetnicki
author: Christopher Skrzetnicki
maintainer: Christopher Skrzetnicki <gtener@gmail.com>
stability: experimental
synopsis: 
description:
category: Other
cabal-version: >= 1.2
build-type: Simple
 
executable dynbin-dyn
  executable: dynbin-dyn
  hs-source-dirs: src
  ghc-options: -dynamic
  main-is: dynamic_main.hs
  build-depends: dynbin, base == 4.*
 
executable dynbin-stat
  executable: dynbin-stat
  hs-source-dirs: src
  main-is: static_main.hs
  build-depends: dynbin, base == 4.*

There is another solution which is much more fun.

HERE BE DRAGONS

The solution is this: build dynamic library and then use dlopen to open it, then dlsym to select and run specific symbol from the library itself. Pretty simple, huh? So let’s get to work.

First lest define a common symbol for calling. We might choose to take something from output from this command:

$ nm -D dist/build/libHSdynbin-0.0-ghc6.12.1.so
         w _Jv_RegisterClasses
00002dd8 A __bss_start
         w __cxa_finalize
         w __gmon_start__
         U __stginit_base_Prelude_dyn
000019f8 T __stginit_dynbinzm0zi0_DynbinziMain
000019a4 T __stginit_dynbinzm0zi0_DynbinziMain_dyn
         U __stginit_hsloggerzm1zi0zi7_SystemziLogziLogger_dyn
         U __stginit_hsloggerzmtemplatezm1zi0zi0_SystemziLogziLoggerziTH_dyn
00002dd8 A _edata
00002de0 A _end
00001ad8 T _fini
000014f0 T _init
         U base_GHCziBase_unpackCStringzh_info
         U base_GHCziTopHandler_runIO_closure
00002dc0 D dynbinzm0zi0_DynbinziMain_main_closure
00001984 T dynbinzm0zi0_DynbinziMain_main_info
00002dbc D dynbinzm0zi0_DynbinziMain_main_srt
00002db4 D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs1_closure
00001934 T dynbinzm0zi0_DynbinziMain_zdfmainzuaOs1_info
00002da0 D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs1_srt
00002d80 D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs2_closure
00001674 T dynbinzm0zi0_DynbinziMain_zdfmainzuaOs2_info
00002d90 D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs3_closure
000016f8 T dynbinzm0zi0_DynbinziMain_zdfmainzuaOs3_info
00002dcc D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs_closure
0000199c T dynbinzm0zi0_DynbinziMain_zdfmainzuaOs_info
00002dc8 D dynbinzm0zi0_DynbinziMain_zdfmainzuaOs_srt
         U getStablePtr
         U ghczmprim_GHCziTypes_ZMZN_closure
         U hsloggerzm1zi0zi7_SystemziLog_INFO_closure
         U hsloggerzm1zi0zi7_SystemziLogziLogger_Logger_con_info
         U hsloggerzm1zi0zi7_SystemziLogziLogger_alertM2_closure
         U hsloggerzm1zi0zi7_SystemziLogziLogger_alertM2_info
         U hsloggerzm1zi0zi7_SystemziLogziLogger_saveGlobalLogger1_closure
         U hsloggerzm1zi0zi7_SystemziLogziLogger_saveGlobalLogger1_info
         U hsloggerzm1zi0zi7_SystemziLogziLogger_zdwa_closure
         U hsloggerzm1zi0zi7_SystemziLogziLogger_zdwa_info
         U newCAF
         U rts_apply
         U rts_checkSchedStatus
         U rts_evalIO
         U rts_lock
         U rts_unlock
         U stg_CAF_BLACKHOLE_info
         U stg_IND_STATIC_info
         U stg_gc_ut
         U stg_upd_frame_info

But this would make the solution even more fun than it is already. Instead let’s take different approach.

We shall export Dynbin.Main.main with foreign export using symbol defined with CPP macro MAIN_LIB. Then we shall use the same macro in dynbin-dyn to call the exported symbol. Both simple and fun! Here comes the code:

dynamic_dlopen.hs:

{-# LANGUAGE ForeignFunctionInterface, CPP #-}
 
{-
 
Required CPP Macros:
- MAIN_LIB: entry point for dynamic library
 
-}
 
module Main(main) where
 
import System.Posix.DynamicLinker
import System.Directory
import System.FilePath
import Control.Applicative
import Foreign
import Paths_dynbin ( getLibDir )
 
main_lib = MAIN_LIB
 
main = do
  libdir <- getLibDir
  print libdir
 
  shared <- filter (\fn -> takeExtension fn == ".so") <$> getDirectoryContents libdir
  case shared of
    [] -> error "No shared libraries!"
    [x] -> putStrLn ("One shared lib: " ++ x) >> onShared (libdir </> x)
    _ -> error "Multiple shared libraries!"
 
foreign import ccall "dynamic" call_dyn_fun :: FunPtr (IO ()) -> IO ()
 
dlopen' a1 a2 = print (a1,a2) >> dlopen a1 a2
 
dlcall :: DL -> String -> IO ()
dlcall dynlib fun = do
  print (dynlib,fun)
  fptr <- dlsym dynlib fun
  case nullFunPtr == fptr of
    False -> call_dyn_fun fptr
    True -> putStrLn =<< dlerror
 
onShared :: FilePath -> IO ()
onShared libPath = do
  main <- dlopen' libPath [RTLD_LAZY, RTLD_LOCAL]
  dlcall main main_lib
  putStrLn "main_lib exited"
  return ()

static_main.hs , unchanged:

import qualified Dynbin.Main
 
main :: IO ()
main = putStrLn "static_main.hs" >> Dynbin.Main.main

Dynbin.Main:

{-# LANGUAGE TemplateHaskell #-}
 
module Dynbin.Main ( main ) where
 
import System.Log.Logger.TH (deriveLoggers)
import qualified System.Log.Logger as HSL
import System.Log.Logger ( updateGlobalLogger, rootLoggerName, setLevel )
 
$(deriveLoggers "HSL" [HSL.WARNING, HSL.NOTICE, HSL.INFO])
 
main :: IO ()
main = do updateGlobalLogger rootLoggerName (setLevel HSL.INFO)
          infoM "Oh hai!"

And this solution works as expected:

$ dynbin-dyn   
"/home/tener//lib/dynbin-0.0/ghc-6.12.1"
One shared lib: libHSdynbin-0.0-ghc6.12.1.so
("/home/tener//lib/dynbin-0.0/ghc-6.12.1/libHSdynbin-0.0-ghc6.12.1.so",[RTLD_LAZY,RTLD_LOCAL])
(DLHandle 0x09216080,"main_lib")
Dynbin.Main: Oh hai!
main_lib exited
$ dynbin-stat
static_main.hs
Dynbin.Main: Oh hai!

The whole setup is pretty fragile. If you build run-dyn without -dynamic flag you will get runtime errors. Note that this behaviour is likely to be linked with this ticket.

Without -dynamic:

$ dynbin-dyn                                   
"/home/tener//lib/dynbin-0.0/ghc-6.12.1"
One shared lib: libHSdynbin-0.0-ghc6.12.1.so
("/home/tener//lib/dynbin-0.0/ghc-6.12.1/libHSdynbin-0.0-ghc6.12.1.so",[RTLD_LAZY,RTLD_LOCAL])
dynbin-dyn: user error (dlopen: 
        /usr/lib/ghc-6.12.1/ghc-prim-0.2.0.0/libHSghc-prim-0.2.0.0-ghc6.12.1.so: 
        undefined symbol: stg_newByteArrayzh)

So, what do you think about the whole idea?


Configuration used:

$ uname -a
Linux laptener 2.6.32-ARCH #1 SMP PREEMPT Tue Jan 19 06:08:04
       UTC 2010 i686 Intel(R) Core(TM)2 Duo 
       CPU T7500 @ 2.20GHz GenuineIntel GNU/Linux
$ cabal -V
cabal-install version 0.8.0
using version 1.8.0.2 of the Cabal library 
$ ghc -V
The Glorious Glasgow Haskell Compilation System, version 6.12.1

I also have two non-default flags in my ~/.cabal/config file:

   library-profiling: True
   shared: True

Sharing wireless connection with wired computer

•07/01/2010 • 1 Comment

Yesterday I wanted to install Arch Linux on a brand new mini computer (with Quad-Core Atom!). I did that on wired connection in one place and then wanted to install additional packages at home.

Problem is: I only have wireless connection at home. So I decided I’ll share wireless connection using my laptop connected to a switch. Here is a diagram of my setup:
Network setup

The hardest part was finding manual how to do this that is on the right level of advancement. There are plenty of descriptions how to create a corporate-level network and configure that with loads of features. And they’re all far too long and quite an overkill for my purposes.

The solution I came up with comes from many sources, most notably Arch Linux wiki pages about basic routing and dnsmasq. The rest is manual pages and various other sources.

So, how to achieve what I wanted?

  1. Enable kernel routing in laptop. Hardly any manual mention that simple step. The temporary solution is to execute command “echo 1 > /proc/sys/net/ipv4/ip_forward”. Alternatively in /etc/sysctl.conf set “net.ipv4.ip_forward=1″.
  2. Install iptables and dnsmasq. Both are needed for this setup. First one will handle forwarding packets from “Laptop sara” and “Laptop router” subnets. The second is a relatively tiny DNS and DHCP server.
  3. Configure dnsmasq. In router’s subnet ip’s are from 192.168.1.1-255 range. dnsmasq will use eth0 interface and assign ip’s from 192.168.2.2-255 range. Here is /etc/dnsmasq.conf that will do that:
    interface=eth0
    expand-hosts
    domain=localdomain
    dhcp-range=192.168.2.2,192.168.2.10,1h
    dhcp-host=sara,192.168.2.3 # static assignment
    #log-dhcp # usefull for debugging
    log-queries # less spam than above, can be left enabled

    Of course we need to enable dnsmasq: “/etc/rc.d/dnsmasq start”.
  4. Configure iptables. Simple and fun:
    iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE # note the use of 'wlan0' here
    /etc/rc.d/iptables save
    /etc/rc.d/iptables start

  5. 192.168.2.1 will be used as eth0′s address. This one is simple: “ifconfig eth0 up && ifconfig eth0 192.168.2.1″. I also had to turn of wicd connection manager daemon, since it used to disconnect me from wired connection from time to time for no real reason: “/etc/rc.d/wicd stop”.
  6. All that is left is making sara ask for assignment: “sudo dhcpcd” and done. We are up and running!
 
Follow

Get every new post delivered to your Inbox.