After some years of mutualized hosting, dev.licorn.org is now living on its own on a brand new Ubuntu 12.04 LTS server. It lives in a dedicated LXC container; the hosting machine is fully dedicated to the Licorn® project, running docs.licorn.org in another LXC, and many more things I wanted the project to have (a build machine, etc).

Meanwhile, we have switched from darcs to git. Having encountered the  fatal #1520 bug has urged us to do so, because this bug just prevents us from working. But it's really the tip of the iceberg, because I wanted to switch to Git for a long time.

Even better, we now officially use  GitFlow to follow the famous  successful Git branching model, and I'm very happy with this clean approach.

Things will only go better now ;-)

The title of this blog post could have been a ticket summary like: WMI takes 100% of CPU when daemonized (with a lot of "WSGI application error" and EBADF crashes), but everything is OK when it stays attached to the current terminal. Luckily I found the solution before posting a new ticket and pushing changes to the branch ;-)

It made me suffer from a headache, because there is no documentation on this subject on Twisted website. Twisted folks tend to think everyone will use their twistd for daemon-purposes, which is a broken assumption when Twisted must just fit into an existing architecture.

I searched a lot on the daemonization side and cleaned my foundations.process.daemonize() function a little but this didn't help, anyway. Then  Rob Golding gave me an hint about it, via a very generic « twisted django wsgi » google search with a very lot of luck.

I just moved the twisted imports after the daemonization call, and everything went fine again. Hopefully, importing twisted.web.version is still possible at the beginning of my wmi.py file without breaking this thread-pool thing, which makes the "please install twisted.web" message still possible too. Nice!

I wanted to know if the WMI2 can handle some correct traffic. It's a management interface, so I don't expect it to undergo real pressure, but we need a minimum.

Testing was performed on my development Virtual Machine, running on my MacbookAir? (late 2010):

cat /proc/cpuinfo

processor	: 1
model name	: Intel(R) Core(TM)2 Duo CPU     L9400  @ 1.86GHz
cpu MHz		: 1860.000
cpu cores	: 2

free

             total       used       free     shared    buffers     cached
Mem:       1536928    1453196      83732          0       8500     101312
-/+ buffers/cache:    1343384     193544
Swap:      1570812     648604     922208

I first tried  Apache Benchmark, but it showed a lot of SSL read failed - closing connection without any reason: all seems OK on the server side. Anyway, the results:

ab -n 1000 -c 50 -r https://localhost:3356/login/

[...]
SSL read failed - closing connection
SSL read failed - closing connection
[ more SSL errors ]
Completed 1000 requests
Finished 1000 requests

Server Software:        
Server Hostname:        localhost
Server Port:            3356
SSL/TLS Protocol:       TLSv1/SSLv3,AES256-SHA,2048,256

Document Path:          /login/
Document Length:        0 bytes

Concurrency Level:      50
Time taken for tests:   35.158 seconds
Complete requests:      1000
Failed requests:        1000
   (Connect: 0, Receive: 0, Length: 1000, Exceptions: 0)
Write errors:           0
Total transferred:      4601000 bytes
HTML transferred:       4075000 bytes
Requests per second:    28.44 [#/sec] (mean)
Time per request:       1757.888 [ms] (mean)
Time per request:       35.158 [ms] (mean, across all concurrent requests)
Transfer rate:          127.80 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:      129 1332 275.7   1381    1896
Processing:    14  387 278.8    373    1171
Waiting:       14  387 278.8    372    1171
Total:        373 1719 302.8   1740    2464

Percentage of the requests served within a certain time (ms)
  50%   1740
  66%   1821
  75%   1908
  80%   1945
  90%   2073
  95%   2172
  98%   2283
  99%   2431
 100%   2464 (longest request)

Reading  this, I decided to switch to  HTTPerf, and it worked a lot better:

httperf --max-connections 50 --max-piped-calls 10 --num-conns 1000 --ssl \
    --server localhost --port 3356

httperf --client=0/1 --server=localhost --port=3356 --uri=/ \
    --max-connections=50 --max-piped-calls=10 --send-buffer=4096 \
    --recv-buffer=16384 --ssl --num-conns=1000 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 1000 requests 1000 replies 1000 test-duration 62.415 s

Connection rate: 16.0 conn/s (62.4 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 39.5 avg 62.4 max 155.0 median 59.5 stddev 12.3
Connection time [ms]: connect 15.8
Connection length [replies/conn]: 1.000

Request rate: 16.0 req/s (62.4 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 13.8 avg 15.9 max 17.2 stddev 1.2 (12 samples)
Reply time [ms]: response 7.3 transfer 39.4
Reply size [B]: header 226.0 content 0.0 footer 2.0 (total 228.0)
Reply status: 1xx=0 2xx=0 3xx=1000 4xx=0 5xx=0

CPU time [s]: user 12.99 system 39.78 (user 20.8% system 63.7% total 84.5%)
Net I/O: 4.5 KB/s (0.0*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

So, these are just bare numbers for a no-real-at-all test, but at least the server responded successfully without crashing, which is fine for a quick test in a development branch. I can continue coding!

On my production server, I noticed that licornd consumes 218Mib of RAM. Granted, this is the stable version (thus, the WMI still lies in licornd) and it holds 214540 directory watches, but I still find this is too much.

I thus gave  Meliae a try, starting from this  perfect example-based howto. Here is the result:

Total 961436 objects, 261 types, Total size = 197.5MiB (207061468 bytes)
 Index   Count   %      Size   % Cum     Max Kind
     0  214540  22 117567920  56  56     548 Watch
     1  466050  48  65864052  31  88    6976 str
     2    2571   0  14768280   7  95 6291592 dict
     3  216166  22   2593992   1  96      12 int
     4    3291   0   1020956   0  97  178172 unicode
     5   23461   2    939256   0  97     908 tuple
     6     338   0    766784   0  98    6304 module
     7    5554   0    377672   0  98      68 code
     8     841   0    376768   0  98     448 type
     9    1864   0    305696   0  98     164 _RLock
    10    5321   0    297976   0  98      56 function
    11     166   0    282200   0  99    1700 Group
    12     508   0    278384   0  99     548 Machine
    13    4474   0    210876   0  99    9336 list
    14     811   0    154508   0  99     548 Enumeration
    15     737   0    120868   0  99     164 _Condition
    16      55   0     93500   0  99    1700 User
    17     557   0     91348   0  99     164 _Event
    18     165   0     90420   0  99     548 Distribution
    19    2765   0     88480   0  99      32 builtin_function_or_method

Seems like  the optimization I submitted to pyinotify is not installed on this machine. A quick approximation suggests we can go down to 130Mib with the patched pyinotify. After having installed it and restarted licornd, I was right. Now licornd eats only 132Mib. Yeah! That's a first 40% cut off :-)

The next question is: how to reduce all these str? They are most probably the pathnames of the watches (given the numbers). A first part of the optimization will be easy: they seem to be stored twice. I'll have to hunt this, but the answer is somewhere in the INotifier. Perhaps we store the pathnames, and so does pyinotify.

The second optimization will probably not be that easy: I assume the best way to reduce the remaining str will be to reimplement pyinotify with a tree (mimicing the FS) to avoid storing the dirname() part of every entry, instead of a list/dict of full pathnames.

This will surely help on the performance side too, in the critical process of comparing new events pathnames against already watched full pathnames as dict keys during watch creation / deletion. Currently, 10 minutes after the daemon launched, there are only 30K watches installed out of 214K. This is poor. OK, my HDD are slow and my machine is 5 years old — and loaded besides of that — but we can do better for sure.

*Morning note:* 7 hours after launch, there are still only 135K watches installed out of 214K. This is very poor. The tree is really needed. This helps explaining why licornd misses inotify events when it's fully loaded with a bunch of watches, and why untarring a kernel on a test machine with nothing watched works like a charm.

History note: I  started from stackoverflow, and  Meliae seemed to be the best tool to use. The nice fact is that in Licorn®, it's ultra simple:

get in
from meliae import scanner
scanner.dump_all_objects('/tmp/licornd.json')
del scanner

And then, after having patched Meliae  against #876810:

ipython
from meliae import loader
om = loader.load('/tmp/licornd.json')
s = om.summarize(); s

After a week-end of work, I finally managed to get Licorn® running on Debian Squeeze. Everything is not yet functionnal. Most notably the ServiceExtension class needs a little more love: because it assumes upstart is installed, which is obviously not the case on Debian ;-).

Basic users and groups management works in the WMI. I still need to test a bit more the whole things, but it goes well via the  developer installation.

From another Licorn® machine running Ubuntu, the « Debian » thing is clearly visible (in the Machines tab of the screenshot) ;-)

Related screenshots:

I've been working hard lately to finish a report summing up my last two years of development on Licorn®. The report is in french because a great part of the development was sponsored by  ADEME (see also  Energy Efficiency in Europe), which is a French organization.

Focuses of the document :

• brief presentation,
• Licorn® structural advantages / weaknesses,
• what we have accomplished in two years,
• the current architecture
• still open developments

If you can read french and want to know more about the internals and externals of Licorn®, this is a must read. The document is far from exhaustive, but gives a good insight of how Licorn® works. If you prefer to learn what Licorn® do, head up to  the documentation website.

Download  the report (PDF, french, 55 pages, 22Mb).

I've finally managed to finish the WMI2 schemas. I created them for the needs of a scientific report, sponsored by  ADEME ( Agence de l'Environnement et de la Maîtrise de l'Énergie; Agency for Environment and Energy proficiency). 2 schemata representing licornd internal architecture, before and after the work partially sponsored by ADEME and  META IT (the company I work for).

Architecture of Licorn®, 1 year ago:

Architecture of Licorn®, as in the WMI2 development branch (undergoing review for merge in the stable tree). This schema is an enhanced version of the one published in the previous blogpost:

The schemata still need to be annontated to be fully understandable. They are not strictly formal in any way. They are just meant to illustrate how licorn works internaly to be understandable by a newcomer. At some time, they will be — and are already, partially — explained on our  documentation website.

The following schema tries to explain how the future WMI2 is working. This architecture is already fully functionnal in our development repository and will enter stable branch soon (official due date: March 5th, 2012). We are currently in the process of polishing it before releasing the patch.

The new WMI will be a huge step forward interactive and web-2.0-like interface ; everything can be handled asynchronously, and the WMI can update any part of its interface without refreshing the whole page. It is even a fully-featured Django + Jinja2 + jQuery application, which runs on top of our new webserver. The webserver is fully WSGI compliant, and built on top of the great gevent co-routine-based library.

As a side note, I've now integrated the WMI2 repository into Trac to help follow the changes and global Licorn® activity. Without this, one could think that Licorn® development is halted, because it all happens in this separate branch, until its merged into the stable one.

This makes team-development easier, by pushing patches very quick between our repos. In your .bashrc, insert this code, and adapt $REPO to fit your needs :

function push () {

	REPO="dev.licorn.org:/home/groups/licorn.wmi2"
	make darcs_record_prehook
	darcs wh -l

	echo -n "OK to record? ([additionnal record message] + [Enter] or Control-C to quit): "
	read -e MESSAGE

	if [ -n "${MESSAGE}" ]; then
		darcs rec -a -m "Work in progress `date '+%Y-%m-%d %H:%M:%S'` (${MESSAGE})."

	else
		darcs rec -a -m "Work in progress `date '+%Y-%m-%d %H:%M:%S'`.";

	fi

	echo -n "OK to push? ([Enter] or Control-C to abort): "
	read DUMMY

	darcs push -a ${REPO}

}

I just found  D3, while researching for various things related to Licorn® evolutions. I'm pretty impressed (the word is weak).

I just experimented a little with it, and the result is very cool. See before:

And After:

What you can't see it that the graph is completely dynamic, made of SVG, and refreshed every 5 seconds with smooth animations, scale morphing, label smart-placing and other cool things. There are still some rough edges (in the way scale lines come in), but this is just an experiment. Which is very very positive, BTW.

Hope you like it.

I've just updated the Licorn® documentation:

• the  modules states are now clearly defined; the code is not yet up-to-date, but it's the next thing to do on core, backends and extensions.
• backends documentation is a little more clean than before. The URL to access it is better spelled and doesn't display a 403 HTTP error anymore:  http://docs.licorn.org/core/backends.
• French translation has progressed a little, on the related subjects.

Yes. Finally, this long-awaited feature is here: you can now attach to an already-forked daemon, and type commands inside it, remotely. Just enter get in (from "get inside") on your prefered CLI prompt.

I have been thinking about this remote debugger for a while. Crunching rlcompleter, readline and a derived InteractiveConsole in my head for days, It seemed really feasible and nearly easy, but I still missed a key to write it. I wasn't far from the result, which I found nearly complete in the  rfoo python package.

Reading rfoo code inspired to me the #645, because it seems to have everything we need, while beeing much more shorter code-wise, and perhaps much faster than Pyro: the hard part of rfoo is implemented in C (via  Cython). When performance problem arise in the cluster, I think we *will* have to switch to rfoo.

The easy part of the switch is that rfoo is "call compatible" with Pyro (both are completely transparent to python programs). Only the PyroProxyWithAttr cannot be implemented in rfoo, which seems wise enough from the security standpoint (everything in Licorn® will be reimplemented via methods to fullfit the security decorators, anyway).

All I have implemented ahead of Pyro seems to be re-usable "out-of-the-box" with rfoo, which makes me think the security decorators I'm thinking about will be, too.

On the road, while reading and derivating rconsole code for Licorn®, I found the fix for #582. I will surely offer myself a beer, or perhaps my collegues will offer it to me, If they read this. But the persons who really owe a beer are  the rfoo developpers.

A big "thank you" to them. They saved me a great time, and showed me how to fix one the most obscure bugs.

NOTE: as a consequence, the local interactive console has vanished from the daemon. We can still interact with it, but this is becoming more and more useless, as much more things are available remotely (get status, get events, get inside), and usable more than once, in parallel, with different options.

This file contains filenames that darcs should not care about. So here we go (only the relevant part), in _darcs/prefs/boring on every of your repos:

^tests/data/scenarii
^tests/data/\..*_status
^tests/data/\.owner
^tests/data/wmitest

docs/_build
^locale/fr
^interfaces/wmi/.*_donnut.png
^interfaces/wmi/js/json

\.pye$
^tailoritself\.
\.mo$

I don't know if it's interesting, but still  a worthly read.

Recently I added 2 new functionnalities to the Licorn® daemon and the CLI interface. The first is a simple extension of the existing get status. There is now a -m CLI switch, which implies that the CLI process stays connected to the daemon, and updates the status on the terminal. The refresh delay and various parameters (fixed refresh count / maximum refresh time / clear screen between outputs) are configurable via other cli switches; see get status --help for details. Just remember this command:

get status -m

Default parameters are cool anyway (refresh time: 1 second, clear screen: yes, refresh count: infinite, max. refresh time: infinite). Just hit Control-C to stop the status updates when you're done.

The second functionnality could be seen as an internal sniffer, in the daemon. It's called monitor mode. With a new CLI command (get events, not to name it), you can access any internal event, decision or message generated in the daemon. This monitor mode will eventually supersede the LTRACE facility, but it's not very clear at this point because the monitor could impact the daemon performance much more than LTRACE (it's always compiled in, whereas LTRACE is usable only when the Python interpreter is in debug mode).

You use the monitor mode like you use LTRACE (levels are the same for both). E.g. to display std (standard) events:

get events

Or to dump only the inotifier related events:

get events -f inotifier

You can combine them to your liking, just the same you would do with LTRACE:

get events -f 'core^system|daemon^threads'

The bonus is the full sniffing mode: just add 'logging' to the facilities you want to follow, and every output message sent by the daemon to any other CLI command (run by you or by others) will be pushed to your local monitor session:

get events -f 'core|logging'

The bonus: you can even monitor a remote daemon from your local machine. Just set the special environment variable before launching the command:

export LICORN_SERVER='IP_or_hostname'
get events

And you're done! Happy monitoring, and remember: these things can easily flood your terminal!

NOTE: currently, only inotifier and logging are fully converted to monitor mode. Other parts will soon follow. The rest of the code still uses the LTRACE facility.

In a few words, on the server side (Ubuntu Maverick):

/etc/rsyncd.conf:

[Sauvegardes]
	# not beiing root seems safer to me.
	uid = rsync
	# the gid is the tricky part and must be forced, else tranfers fail. 
	# rsp-* implies the daemon on the server side has always full permissions.
	gid = rsp-Sauvegardes
	path = /home/groups/Sauvegardes
	comment = Sauvegardes
	read only = no

/etc/default/rsync:

RSYNC_ENABLE=true
RSYNC_OPTS='--address=192.168.111.1'
RSYNC_NICE=''
RSYNC_IONICE='-c3'

in the root shell:

# not that needed, but seems clean to me
add group --system rsync

# this one is really needed
add user --system rsync --force

# not that needed too, but seems clean to me
add user rsync rsp-Sauvegardes

On the client side:

rsync -a <other options> dir1 dir2 ... leto::Sauvegardes/

Speed note: my server Leto beiing a "poor" and old monocore AMD Sempron, using rsyncd permits to speed up the transfers up to the client disk reading speed (roughly 25Mb/s in best moments) using only 30% CPU, instead of being CPU clamped to 100% with ~8Mb/s maximum speed when wrapped inside SSH (which is totally useless on a trusted local network, IMHO).

You may have noticed or not, but recent changes in the core lead to enhanced console output for get CLI tool. We use utf-8 as much as possible, to include icons and special characters and colors which give a bunch of informations without cluttering your terminal with things to read.

Thanks to the recent core rewrite, we have now high-level methods and properties for building these output informations, and get has an internal cache / invalidation mechanism to avoid wasting CPU cycles.

I'm currently reading a nice  TS-framework-related article, and some others (search for " nose versus py.test with google"), and I'm seriously thinking about (sort of) getting rid of, or enhancing a lot our home-made testsuite.

Sure, our testsuite has advantages of very-small-scenarii writing, very-high-level (command combinations, varying context, etc) testing, auto-included teardown commands and much-much-beautiful output (very human friendly IMHO). But we now need low-level unit testing in Licorn®, and the TS can't handle this without a consequent rewrite.

There is currently no way to test something as simple as a method raising an exception when given bad input, and I miss this. BTW, now that our code base is increasing fast, I'm looking for a way to split core.py into smaller files, and spread the test methods in the relevant part of Licorn®. Neither to say that auto-collecting test methods would be more clever than needing to chain them in core.py.

Perhaps we will acheive some sort of combinations between the two worlds. Beiing capable to test things in the background while hand-checking a current failure is something I'm not yet ready to loose.

The new ConfigurationFile class (currently beiing written and tested, thus the research and this blog entry), based on  the pygments parser is something that needs to be very-very carefully tested before use, and the current TS can't do this easily.

More to comes in the very-near future, both works are progressing alongside.

PS: additionnaly, have a look at  django specific testing. Generaly speaking,  This might me a worth read, too.

With #500 implemented, Licorn® now supports mounting any kind of volumes on the local machine, provided you add extensions.volumes.mount_all_fs = True in your /etc/licorn/licorn.conf file.

This makes licornd provide the same functionnalities as udisks, allowing to avoid it completely when you run licornd on your laptop or desktop machine.

End-User-formatted (vfat/nfts/iso9660/udf) volumes will be mounted with the UID of the console user. Other types (ext*/btrfs/reiserfs/xfs/jfs) will be mounted the standard way, with acl and user_xattr options.

If for any reason you plug your device in before logging in on your machine, just launch the following commands:

del vol -a
add vol -a

And licornd will remount the volumes accordingly for you.

Note: I didn't test the behavior when logged in remotely. I don't think an ssh remote login will be treated like a normal local login. This will probably be implemented in a future enhancement if needed.

Every part of Licorn® can now emit events, and any other part of it can setup a callback to receive the event arguments, to do what is needed. Events callbacks can be run synchronously (with method L_event_run()) or not (with method L_event_dispatch()). In this case, there is no guarantee of callback order call, jobs can be parallel because they are handled by the service facility. Example:

Emitter side:

from licorn.daemon import priorities, InternalEvent

# L_event_dispatch is a builtin, defined by the daemon at start
# no need to import anything, to use it.

L_event_dispatch(priorities.NORMAL,
                 InternalEvent('main_configuration_file_changed'))

Receiver side:

[...]

def main_configuration_file_changed_callback():
    """ this method will be auto-collected if the surrounding object 
        is a controller, or a CoreUnitObject whose controller defines
        self.__look_deeper_for_callbacks (typically ModulesManager 
        does it). """
    #
    # do whatever needed with LMC.configuration, whose
    # contents already reflect the change.

The EventManager is ready to operate even before the INotifier, and just after the LMC is setup. It will collect all callbacks of the controllers, backend and extensions on start.

It is stopped last, at the end of the daemon life.

Note: the InternalEvent instance can be given a callback argument (any callable() can apply for the job), to resync the Emitter at the end of event callback execution.

I'm proud to announce the new inotifier rewrite (and its bunch of small enhancements), internally and lovely named "hopefully-this-one-will-work-as-expected" (private joke to me). It's shorter than previous version in terms of codelines, albeit more complex when dealing with special cases (large directories, multiple concurrent accesses to same files, re-born just-deleted files or dirs, etc). The new version is many times faster than all previous ones (including the external C-implemented gamin one). When you untar an archive, you can expect more or less the time of the untar process, after it finished, for complete ACLs application. Previously, it could take minutes to do the same (specifically when untarring the linux kernel in a shared dir). licornd is also very smart when talking about resources-consumption: it takes the CPU for ACLs intensive tasks (but only ONE CPU), and doesn't take it long. For what it has to do, I find it well balanced from the functionnality/resource point of view.

The new inotifier and related core.classes additions allow users homes to be watched now, and offer dedicated functionnalities to handle configuration files, and report *real* changes to them (not 'all access', generating a lot of false positives).

dnsmasq backend, privileges directly benefit from this new functionnalities. shadow configuration files watch is more robust and verifies everything when they reload (one could create inconsistencies, editing the files manually; this is taken in account).

There are still some rough edges and evil sub-sonic bugs (perhaps they are all the same, I can't hunt it down for now), but only on very-very heavily loaded systems, where users and groups pop in an out very fast. I will fix them in the next coding cycle.

Hopefully, you won't need the chk group command anymore. If you do, please provide a full trace:

export LTRACE=std
licornd -rvD

<whatever command in your other terminal>

In the new-but-small-but-cool features category, you'll find the command fuzzy matching:

get u
get us
get usr
get users

(and so one, with identical counterparts for add/mod/del/chk)

Will bring you the list of users. In the same kind:

get g     -> groups
get pro   -> profiles
get pri   -> privileges
get kw    -> keywords

And so on. Everything is computed when you type it, there are no so-called "fixed values".

In the not-so-small-but-very-cool category, you will find that every part of Licorn® is now fully multi-lingual, on-the-fly: the daemon starts in the system lang, but every thread inside of it can switch to another language, and the client languages are pulled in from the web headers or the calling CLI environment. This makes everything dynamic, at will.

Documentation has been updated for permissions parts.

French tranlation is progressing notably: WMI part is finished, CLI is 90% done, and the rest is more or less 70% done (it doesn't matter anyway, as no user really sees it in real life).

I voluntarily don't mention the core object rewrite. It's very technical and doesn't bring new end-users functionalities, but guarantees that everything is cleaner and easier to extend inside licornd, regarding the users/groups/profiles/privileges/machines point of view.

I probably forgot many things here, but if I had written a book, you won't have read it anyway. Code and *use* the code is better. Many bugs have been fixed, and the code is generally more pythonic and lighter tht before: there are more generators, less hard-coded things, and abstractions (when necessary) got in the right places. At least, this how I wanted to implement them.

Enjoy,

The past 2 weeks have featured a great core rewrite that I wanted to achieve for a long time. Core objects (Users, Groups & Profiles) are now clean objects, implemented with all the pythonic-fancyness that modern code can have (most notably properties, weak references and internal generators where applicable). Controllers manipulates them in a clean way too, doing things the CoreUnitObject can't do because they are not aware of the controller context (which make the whole thing totally logical, finally; things as they were meant to be, at last; [put your favorite self-satisfaction sentence here]).

Controllers and unit objects moderately use the daemon service facility, to make things more instant to clients and avoid long-activity stoppers: for example, setting permissive ON/OFF on groups launches a background check on shared data and returns instantly (among others).

As a consequence, the service facilities are initialized very early in the daemon (even before the LMC initializes) and are usable everywhere: Each individual object has a licord R/O property (named after this avoid collision with the Thread's daemon attribute), offering directly the {service,aclcheck,network}_* methods. The imports overhead and dirtiness of the previous implementation is totally avoided.

The patch lies in the  development repository (not referenced in trac but accessible to SSHers). I will not push it to the stable branch until the new inotifier has landed, but it is very stable (testsuite has run many times on it).

The Full-i18n milestone will soon be closed (or nearly), because #2, #70, #541, #542 and #544 are implemented or closed in this patch. Thus, besides the full-object rewrite, we now have on-the-fly in-thread language transparent switching, and this really rocks for the WMI.

Starting from now, the only thing you've got to do to enable LTRACE is to set the environment variable. If the daemon finds it at start, it will refork itself with the appropriate options to enable it.

The LTRACE status is maintained while restarting from inside the daemon (this was not as easy as it seems to implement).

Bonus: just export LTRACE=std to get a nice but not so verbose LTRACE output, you will notice that the only places where the daemon hands on stop/restart are justified (__unihibit_udisks() and PyroFinder queue not empty). In this case, just wait. The udisks thing can take up to 10 secs, and the PyroFinder one, up to 20 secs (i'm searching how to lower this one, the Pyro timeout is already set to 5 secs).

Theses extensions are ready for wider testing. They have their documentation setup ( volumes and  rdiffbackup).

A major thread rewrite has been done during this dev cycle.

CLI and WMI related parts are coming in.

Tonight I sprint-coded the  OpenSSH extension, removed all openssh-related code in LMC.configuration, and added system-service-related code to  SystemController class.

Guess what:

• It works like a charm :-)
• I readded the missing functionnalities from previous releases (the AllowGroups configuration directive), and added much more (enable/disable ssh extension and service, check other configuration directives, verify the sshd is running...)

By the way, members of remotessh are allowed to connect, and members of admins too (regardless of their remotessh member status: this is cool because you (the reader) are probably already a member of admins, and you won't ever notice the sshd group-authorization enforcement ;-)

The  OpensshExtension module can serve as an extension model: it seems quite clean (and short) to me.

This work would not have been so easy without the ConfigFile class Robin wrote. I enhanced it a little, but it's so practical!!

To avoid conflict with eventual future backends and to stick a little more with reality, the unix backend has been renamed to shadow, and the ldap backend has been renamed to openldap. They are held in files shadow.py and openldap.py in the ShadowBackend and OpenldapBackend classes. Much readable, isn't it?

Configuration, testsuite,  documentation website are up-to-date with this change.

Module loading is now much more clean: the backends and extensions do not need to be instanciated in their python file anymore, this is taken care by the manager. This makes code much understandable (I hope), shorter, and more standard. Sphinx autodoc can now be used on backends classes too, whereas it wasn't possible before.

core.backends.classes has dissapeared (the related code is now directly inside core.backends.__init__), and idem for extensions.classes (now in extensions.__init__).

So, The python modules filenames and controller classes names changed, and are shorter: e.g. dnsmasq.py holds the DnsmasqBackend (instead of dnsmasq_backend.py containing dnsmasq_controller), and so on. This rule applies for backends and extensions.

Small other renames have landed too, like ModuleManager having becomed ModulesManager and other minor changes.

The daemon has now an auto-discovery capability on its local network(s). It will scan the LAN (on all of its ethernet interfaces) and will try to discover all hosts which are up (answer ping).

With  10 to 30 network threads in each pool (the default beiing 5, is resource-conservative), it can be faster than nmap to do a full LAN discovery (you will gain ARP resolution and Pyro resolution bonuses).

You can  disable the feature if you don't like it or experiment problems with it (don't forget to report bugs in this case).

We finally got rid of nmap and all those consuming subprocess.Popen calls, with a big functionnality gain.

Thanks to Robin who did all the hard work, chk can now be customized for users. This allow to avoid ACLs for a certain file hierarchy, or force some custom ones on another. For instance, I use these, in my ~/.licorn/check.conf:

source	NOACL
build	NOACL
Projects	NOACL

This allow debian tools not to crash when building packages or scanning source trees.

There is a little loss in performances, but the win is clear compared to before. We will optimize the code on next performance run (there is a dedicated ticket for that).

System ACLs rules can be fully customized (with only a few sane exceptions) by administrators. For more details,  there's a bunch of related documentation!

We set up  a new site dedicated to Licorn® documentation, growing quickly as we link existing docstrings to it.

Some existing doc from the Trac has already migrated there, and brand new doc will be written, related to profiles management, group memberships, data sharing models between members of a same group, and the like.

Configuration files and directives will be documented there, too.

See you there, keep up bringing peace to the Earth.

You can debug licornd in a live interactive session. Just start it in the foreground (licornd -D), and press 'i' . You can access every object and do whatever you want (trigger a method, dump an object, really whatever the python language allows you ; so be carefull, you're root...).

Bonus: everything is auto-completed with <TAB>, like any interactive shell can be, and your command history is saved to ~/.licorn/licornd_history (thanks to readline).

Press Control-D when you are done to leave interactive mode and return to standard command mode.

The interactive session is implemented in a separate thread. The licorn daemon stays fully functionnal during it. Stopping and restarting individual things manually will probably come in the near future, but as of now if your display gets corrupted with other daemon output, just hit Control-L to clear your screen.

Example session:

olive@desktop-001 ~/licorn @ licornd -D
 * [2010/04/12 01:01:44.8412] licornd/master@server(5124): starting all threads.
 * [2010/04/12 01:01:44.8454] licornd/wmi(5129): started, waiting for master to become ready.
 * [2010/04/12 01:01:44.9980] licornd/master@server(5124): all threads started, going to sleep waiting for signals.
 * [2010/04/12 01:01:45.4224] licornd/wmi(5129): ready to answer requests at address http://localhost:3356/.
 * [2010/04/12 01:01:47.6714] Entering interactive mode. Welcome into licornd's arcanes…
Licorn® @DEVEL@, Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) [GCC 4.4.5] on linux2
licornd> LMC.users.keys()
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 33, 34, 38, 39, 41, 300, 113, 100, 101, 102, 103, 1000, 105, 106, 107, 108, 109, 110, 111, 112, 104, 114, 115, 116, 117, 65534]
licornd> dqueues
{'pyrosys': <Queue.Queue instance at 0x9c7a18c>, 'reverse_dns': <Queue.Queue instance at 0x9c7a04c>, 'arppings': <Queue.Queue instance at 0x9c71eec>, 'pings': <Queue.Queue instance at 0x9c71dac>}
licornd> 
 * [2010/04/12 01:02:24.0383] Leaving interactive mode. Welcome back to Real World™.
 * [2010/04/12 01:02:25.3743] licornd/master@server: signal 2 received, shutting down…
 * [2010/04/12 01:02:25.3748] licornd/wmi: signal 15 received, shutting down…
 * [2010/04/12 01:02:25.3751] licornd/wmi: exiting.
 * [2010/04/12 01:02:25.5468] licornd/master@server: exiting (up 40 secs).

PS: i really love Python. These kind of things are just amazing.

PS2: the implementation can be incomplete, I didn't really test every object. Just report any bug you find and I will fix it ASAP.

Starting from now, ltrace understands binary exclusions. You can debug the daemon and CLI tools with better cherry-picking of trace messages:

export LICORN_TRACE='core^configuration^groups'
sudo python /usr/sbin/licornd -vD

This will trace everything in the core, except configuration and groups messages.

Binary inclusions still work, and you can combine the two:

export LICORN_TRACE='foundations^base^objects|core^configuration^groups'
sudo python /usr/sbin/licornd -vD

Which will debug same level of core, but added foundations, without base and objects modules (which are very verbose and very low-level).

NOTE: the order matters when you write components name in the variable:

• you can write 'core^users^configuration', it will work because users and configuration are subparts of core.
• 'configuration^users' will display configuration messages only (users is not a subpart of configuration, nothing more is excluded).
• 'foundations|core^base^users' will work as expected.
• 'foundations^users|core^base' will not substract users from core, because core is processed later by the ltrace parser. base will be substracted from foundations, though.

Defining LICORN_SERVER environment variable (in latest stable code) provides an easy way to test bi-directionnal communication between 2 or more Licorn® daemons installed on distinct machines.

WARNING: this is meant for testing and development purposes only. In production environments, there should be a DHCP server on the Licorn® machine with the SERVER role, and the CLIENTs will discover it automatically.

To use:

• be sure all machines are on the same IP network.
• set up one of the machine with licornd.role = SERVER in /etc/licorn/licorn.conf.
  • find the IP address of this machine and remember it.
• set up all other machines with licornd.role = CLIENT
  • open a Terminal on each, and run export LICORN_SERVER=<IP_address> with the IP address of your server.
• start all licornd, ending with the server one (else it won't detect the clients because they currently don't push their status to the server).
  • alternatively, if you want to force detection of new clients started after the server, you can run add machines --discover <IP_subnet/mask> on the server, it will scan the subnet and register all new machines.

Besides this, be sure to set experimental.enabled = True on the SERVER machine, to get the Machines tab in the WMI.

You can now enjoy remotely shutting down client machines.

Just run mod user -C and you're done.

root can change any user's password.

Any user (member of admins group) can try to change any other user's password, but must know the current one...

A standard user can't change his/her password yet (this needs a full rewrite of the security model), but this will come in the near future.

As you can read in my last big patch (~rev 593), the Licorn® core has been greatly refactorized. There is now a global (but fine grained locked) object, called the LMC (LicornMasterController), which holds all controllers (UsersController, GroupsController and others). It holds 2 other special controllers:

• the BackendController (real object pretty name: LMC.backends) which has been pulled out from the LicornConfiguration object and holds all backends. The news is that all backends are now equal (there is no level difference between dnsmasq and ldap for example). Backend objects are just much more clean to read and understand. Controllers can use different backends if there's a good reason to. A direct consequence is that LMC.configuration is much simpler than before (and the diet will continue).
• the LockManager: it is yet just an Enumeration but will become soon a more clever object. It holds all locks for all other controllers, and all locks for controllers unique records. Locks are stored outside of controllers because Pyro can't pickle lock objects (which seems quite fine if you think about it). The LockManager (real object pretty name: LMC.locks) is not exported via Pyro, because all other controllers use it internally and it is not meant to be accessed directly. To lock an object remotely, just call <controller_name>.acquire() and release() methods as you would on a normal lock and the LicornCoreController class will wrap everything for you.

The LMC is accessed directly in the daemon, and remotely via a single LMC.connect() call, in the CLI and the WMI. Pyro related changes and their consequences in the project are nearing end, and the Licorn® core has benefited a big clean-up from it. This is a big piece of work, and a great enhancement: we gained more functionnalities, guaranteed consistency, fine grained locking, with a much simpler codebase. Let the magic (and hard work) continue.

In my last patch I added the availability to query daemon status, in two ways:

• the get status command, which can be called with argument --full. This command is complemented by get users --dump, get groups --dump and so on, which help debugging daemon internal data structures without stopping it. This method for getting the daemon status is independent from its state (forked into background or not).
• when the daemon is attached to the terminal (launched with -D), you can now type uni-letters commands to query it:
  • 'f' or 'l' will toggle between normal and full status.
  • [Enter] will just display a newline (usefull for manually marking spaces between different operations.
  • Ctrl-L will clear the screen, like in a normal terminal.
  • Ctrl-T will display the current status of the daemon (full status depends on wether you activated it before or not, beiing disabled by default and remembered across the daemon session when you set it, until terminate or restart).
  • Ctrl-Y (or space) will do the same, but will clear the same first. Typing repeatedly on space will emulate a top-like behaviour, permitting to monitor the daemon status, even if it is very active.
  • Ctrl-R will reload the daemon (by sending it an USR1 signal). Very useful when you modified daemon or core code, just hit ^{R in your daemon terminal and you're done with the new code reloaded.}
  • Ctrl-C will break and terminate, as expected.
  • Ctrl-U will terminate the daemon with a traditionnal signal 15 (similating a normal kill or killall).
  • (Caution) Ctrl-K will send a real KILL signal, when the daemon is stuck.

In some rare cases (when the interactor thread is crashed, which never happens ;-) ), you will not be able to use these commands and will need to operate "à l'ancienne" (Ctrl-Z, bg, sudo killall -r licornd and al.).

There's a new experimental.enabled directive available in /etc/licorn/licornd.conf (no command line argument equivalent), which defaults to False if unset in configuration file. This flag will be used from time to time, to test experimental functionnalities in stable branch. Currently this enables the Machines tab in WMI, if you manually set it to True in your local configuration file (which I recommend if you're a developper or an advanced system admin).

Pyro work has been commited to the stable branch. News:

• you don't need to use sudo anymore with licorn commands. If you're a member of group admins, everything should be transparent for you.
• every CLI tool and the WMI needs the daemon to work.
• CLI tools will launch the daemon if needed and wait for it to be ready before continuing.

Functionnaly-wise, nothing should have changed (this is guaranteed by the testsuite).

Security-wise, core objects are not yet protected with locks, but this is the next work to do in turn. For everyday use, this should not hurt (i've tryed to crash it, but didn't succeed).

Because of pyro changes introduced in development branch, we cannot use configuration.daemon.* directives, becase pyro uses the daemon keyword as a reserved attribute (this will change in Pyro 4.x, where all attributes are prefixed with _pyro, but 4.x is not ready yet).

I've thus renamed configuration.daemon.* to configuration.licornd.*. Please change all of your configuration files accordingly.

on Server, install code? from the special repo dev.licorn.org:/home/groups/darcs-Licorn.dev.

On ALT client(s):

sudo apt-get install licorn-server licorn-client
sudo apt-get remove --purge licorn-ldap-server    #until #331 is fixed
sudo apt-get autoremove --purge
sudo add user --system alt
sudo mkdir ~alt/.ssh
sudo cat >> ~alt/.ssh << EOF
<ROOT's_PUBKEY_on-your-server>
EOF
sudo chown -R alt: /home/users/alt   #until the #322 workaround is fixed.
(root@server: ssh alt@alt to avoid the 'fingerpring message')
sudo add users alt admins
sudo vim /etc/sudoers and add `NOPASSWD:` for `admins`.

Then your ALT(s) should be shutdownable from WMI and CLI.

With #190 fixed, you can now do:

get user olive -l
get user 1001
get group root
get group root -l
[and so on...]

This is roughly and equivalent of the venerable id, with Licorn® additions.

Welcome to Robin, a new commiter on the Licorn® tree. He will work with us for 6 months, mainly on Licorn®, but will also meet many other parts of our work ouside this. His work can be followed along on his log page.

The WMI listen address is now configurable, either:

• in the main configuration file (/etc/licorn/licorn.conf if you didn't change it), with the directive daemon.wmi.listen_address. Don't put any quotes around the address. Example:

  daemon.wmi.listen_address = 192.168.1.1
  

• on the command line, when manually starting the daemon. Example:

  sudo licornd -vvD -W 192.168.1.1
  

You can use an IP address, or a hostname (it will be resolved to an IP address). Note that the CLI argument has always precedence over the configuration file, for obvious reasons of easy testing.

With changeset 332, the testsuite results are now stored directly inside our source code repository. This will avoid rechecking the 500+ commands from scratch (*very* time consuming, we must check all return codes and every command output) when running the testsuite on a new source environment.

Note that there will surely be errors reported when you run it, due to configuration differences between my reference machine and others, but this hits only a few commands, so having the whole data in the repository is better than start from scratch.

a bunch of bugs later, the new release has come. Full LDAP support, with many bugfixes. It should be stable, LDAP-wise. I added a few more functionnalities, coming from other milestones.

You know where to report bugs. Don't forget to read the Installation How-To?, and the Debian package dependancies, to know what to install. Salute!

I filled a bug (#221), just to remember about a particular problem in the multi-backend context. It is curently harmless, I can't see any situation where it could do some damage.

After having used bzr for a few months, we switched back to our beloved darcs. Bzr is not bad per se, but darcs is much more smooth, easy and powerfull when it comes to merges, which is what we do most of the time: small corrections of small bugs, pushed and pulled from/to dev branches. The concept of branches in bzr is a little bit more, we can't forward patches on branches with uncommitted code, which is a pain (darcs does this magically if the patch concerns untouched files). The concept of selective pull/push is a great help, too.

I used the pretext to upgrade Trac to 0.12 (migrated from 0.11.7 which was in production only since a few days, migrated from 0.11.1).

Everything seems to work very fine (at least in Trac), but I haven't commited since darcs migration. I will do in the next hours.

I've setup a new page describing the DEBIAN dependancies for the Licorn® package.

Responding to changeset 279 and explaining the reasons of changeset 304, there is now no more limitations when creating system groups.

If the LDAP backend is enabled, the newly created system group will go into the LDAP backend.

This makes NFS works perfectly on LDAP-enabled clients, which now see all groups when listings ACLs and standard posix perms.

Now, the licorn-ldap-server debian package tries to move every important pre-existing system group (acl, licorn-wmi) from the Unix backend to the LDAP one. This will implicitly install them network-wide and avoid the need to create them on every ALT®. Finally this will help propagate admin privileges on every client.

If you know about privileges, you know you can manipulate them only via the configuration object, like this:

get config privs
sudo add group licorn-wmi --system
sudo mod config --add-privileges licorn-wmi
sudo add group remote-ssh --system
sudo mod config --add-privileges remotessh
sudo mod config --del-privileges licorn-wmi,remotessh
get config privileges

Now you can handle them kind of "directly", like this:

get privs
sudo add group licorn-wmi --system
sudo add priv licorn-wmi
sudo add group remote-ssh --system
sudo add priv remotessh
sudo del privs licorn-wmi,remotessh
get privileges

Which is quite simpler, and - I think - more logical or consistent with the rest of the command set. The code lies in changeset 310.

two points which were pushed in revision 279, because bzr don't know anything about cherry picking (or I don't know how to use it):

• system users/groups are now always created in the Unix backend. this is far from perfect because they have to be replicated for ACLs to work and this will NEED to be done for the cluster, but the cluster doesn't exist yet, so don't bother. This is needed for groups like acl, remotessh and admins, which NEED to be created in /etc/group on the local system before the LDAP backend is activated (they are checked/created before the LDAP setup). This is kind of a workaround because it should work event with these groups in LDAP, but this implies some deep reorganization in the LicornConfiguration class, which will be done later (see #170).
• add a --force argument to add user/group, for the particular case of user/group with same name. This was trigerring a bug on old debian/ubuntu systems, but should be bypassable nowadays, because I expect adduser tools to have been fixed on this particular point.

The first version of LDAP backend is out, numbered Licorn 1.2. On a freshly installed Lucid with an empty slapd, Licorn® will do everything needed to get a fully usable LDAP system (pam-ldap and nss-ldap wise).

Just run the following command:

sudo aptitude install licorn-ldap

And let the magic begins. What it does, summed up:

• install slapd, lib*-ldap and all licorn.
• wait for slapd to be started with the bare minimum configuration.
• setup all the rest of the slapd configuration (chk config -avb; read the gory details on the LDAP backend page).
• enable the LDAP backend on the whole system (mod config -b ldap), including lib*-ldap.

PAM setup should be done by the debian packages, which is the case nowadays on Lucid.

To be sure the LDAP module is available and enabled, just ask:

sudo get config backends

And the system should tell you:

ldap*
unix

Everything else is just transparent: you can use Licorn® CLI tools as usual.

There is still work to be done, but it concerns new features only; the LDAP backend is currently feature complete, regarding users and groups.

I didn't expect that, but new configuration system (cn=config) in slapd and differences in distribution version (karmik and Lucid) prevented me to complete the auto-configuration for LDAP backend (autoconfiguration can be tricky if you want to make it right).

As of now the 'pure backend' (users and groups management) works perfectly over LDAP, provided that you manually setup the slapd frontend ("o" and different "ou"'s). You can find out every detail on the LDAPBackend wiki page. There is more information than you need, because the wiki page expects that you start from a Karmak setup (totally empty), whereas in Lucid the module and backend parts of slapd are already present.

Samba part is not yet implemented, but will come later. It needs several things to be worked out before.

Only 7 working days (with standard working hours !) were needed to implement the biggest part of the LDAP backend and fix blocker bugs. It works completely today (get/add/mod/del, wouhou !).

The only work not yet done is slapd configuration, which is not strictly the backend functionnality, but I want it done: Licorn has to autoconfigure slapd when it's freshly installed. Typically on a Lucid Lynx system (Ubuntu), cn=config is completely empty, which render slapd totally unusable, which is not acceptable for us.

This could be resolved by packaging work. But I want Licorn to be autonomous on different system and not duplicate packaging work among distros. By the way, Licorn must in the end be able to change the LDAP base_dn and other configuration attributes. Even though this functionnality is less than autoconfiguration, autoconfiguration is not far behind (the way that I see it, in KISS technology).

As I primarily code new features and bugfixes, I tend to do QA changes when I can. This makes the global code quality improve slowly:

olive@Gany ~/licorn @ find . -type f -name '*.py' | xargs pylint

[...]

Global evaluation
-----------------
Your code has been rated at 3.75/10 (previous run: -1.90/10)

Locally, the quality can be much better. There are files where I spent time to experiment with pylint remarks, and this shows:

pylint core/internals/users.py

[...]

Global evaluation
-----------------
Your code has been rated at 8.32/10 (previous run: 8.17/10)

And files where I already modified some portions of code, but not all (since I don't have time):

pylint core/internals/configuration.py

[...]

Global evaluation
-----------------
Your code has been rated at 6.01/10 (previous run: 3.19/10)

Anyway, major QA work will occur: we have a dedicated milestone for this !

I released the version 1.1 of Licorn®, which is quite a good news. The first release after nearly a year of silence and 2 weeks of code.

All WMI features are completely functionnal, except the "import users" one, which is not critical in early deployments.

LDAP work has begun and I plan to release the first beta version (managing users and groups) between the middle and the end of the week. LDAP support will be very preliminary: we will support only all *nix features, not the SAMBA ones. But ongoing work will take care of extending these.

First goal is to make Linux clients authenticate against a Licorn® server, making the whole thing rock solid. I hate bugs ;-)

I've resolved a small bunch (?) of issues in the WMI. Some are still open, but I plan to close them in the present day. I can smell release :-)

Milestone completed. 11 official bugs fixed, some other (not referenced) fixed too. QA work has started. DEB packages are coming.

WMI is quite usable now, but milestone 1.1 will make it feature complete and rock solid. Work continues !

I re-ordered and renamed milestones, to make them stick better to real life. All new bugs have been accepted.

I'm now using bzr with quite comfort, and have taken back full use of geany and regexer, which I haven't used for a year (I never stopped using GVim, even under Windows and Mac OS X ;-) ).

Starting bugfixing now. In fact it has already started, since I merged external bugs reports before working with trac.

The first pylint pass was a bit of depressiveness: my code rated at -7.7/10, which is quite bad. After some relatively easy work (configuring pylint for my naming conventions, and to use tab as indenter method), the code reached 2/10.

Then I worked on 3 files which didn't have much errors or warnings, just to familiarize with pylint and to be able to reach a result quickly, seiing this work as a learning period.

I was successfull : individual code quality on these files raised from 2/10 to 8.5/10, 9.5/10 and 10/10. Opening these files now in the editor shows a real change in terms of readability and global consistency.

Now that I'm familiar with pylint, I will stick to clean-coding as much as I can, and will operate a major quality re-read and re-code in a future milesone.

I'm back at Licorn® coding after nearly a year of sleep. Many things have evolved since then and I'm coming back with shining new ideas.

First I will merge the new WMI, which has fewer bugs and is more pretty than the old. I will be learning bzr during this period (branching, pushing, merging).

Then I will improve daemon internals, execute pylint and fix bugs.

Then fully implement the keywords part (internals and GUIs), in parallel of other tasks (client, cluster...). Not a prioritary task, but this will add a killer feature, IMO.

The Clients code is coming in. It will handle network hosts (machines, printers…), will make them sticky in the DHCP server configuration files, will push upgrades onto them and make them standby or shutdown at given times, and all that sort of things.

Clients are identified by their MAC addresses, and a Client can have more than a MAC. When new MACs pop up on the network, you will be able to attach them to already existing hosts (when you change the network card, or the mother board of a machine and want to get back the old configuration).

Yesterday and this night I created the basis for a dynamic backend system for users and groups. The Unix files backend has already been externalized from licorn.core.internals to licorn.core.backends.unix and is feature-complete. The LDAP backend is slowly making its way into the code.

Anecdotically, I acheived some more small typo changes and clean-ups in the code.

Today, I've cleaned the daemon's main a lot, moving things to daemon.core and shortening existing code.

I added a configuration directive daemon.role, to prefigure the differences between Licorn® servers and clients. The documentation is updated accordingly.

In r108 I have completely reorganized the daemon's internals. Now the code is somewhat cleaner, files are shorter, and the internal arch is hoppefully easier to understand. Much more will be done during the refactoring period, this is planned for a dedicated milestone.

Note: the configuration directive “daemon.start_wmi” has been renamed to a more suited “daemon.wmi.enable”, to match internal arch.

Note: the new daemon is strictly compatible with the old. Nothing other than the presently stated things changed.

The main configuration file has been renamed from /etc/licorn/main.conf to /etc/licorn/licorn.conf. It can contains directives in any order with any sub objects, like this :

• daemon.start_wmi=False

This will create a configuration.daemon.start_wmi boolean. BTW, #36 is fixed if you set this directive ;-)

Ideas came from #73 : the dense syntax add user user1 group1 (inherited from Debian adduser tools) made me think of extending it a little bit. You can now type :

• add user user1,user2 [--system] : this will create 2 [system] users with autogenerated passwords.
• add user user1,user2 group1 : make the two users members of group group1.
• add user user1 group1,group2 : make user user1 member of the two groups.
• add user user1,user2 group1,group2 : make the two users members of the two groups.

And it will work as expected. The same syntax works with the del CLI tool too. Happy admin !

I converted the Licorn® source repository to Darcs2 format. This will avoid some strange conflicts encountered in the past with my many dev branches. Please send me any awaiting patches in the diff format, because versions 1 and 2 of darcs repos are not compatible (you will not be able to push / pull).

The Licorn® Trac has been updated to version 0.11.2.1, to follow security branches more easily, to have Debian packages installed instead of SVN eggs (thanks to PPA, we can find pretty any software for any Ubuntu version nowadays…), and to be able to use plugins (which this Blog engine is part).

During the Trac migration, I configured default system encoding to utf-8 in Python's site.py configuration file. It's been a long time since I wanted to do this !

There are new handful functions in pylibacl 0.4. Implemented in C, they run surely more quick than the ones in the fsapi module. I updated licorn code to use one of them, which will improve performance of chk command.

Licorn® Development Blog is a great human-friendly addition to the classic timeline. You will find easy-to-read news about all important things happening to the code and the developers. All-for-the-geek stuff. A must-have !