rss
D3 and consequent graphical updates in the WMI
I just found D3, while researching for various things related to Licorn® evolutions. I'm pretty impressed (the word is weak).
I just experimented a little with it, and the result is very cool. See before:
And After:
What you can't see it that the graph is completely dynamic, made of SVG, and refreshed every 5 seconds with smooth animations, scale morphing, label smart-placing and other cool things. There are still some rough edges (in the way scale lines come in), but this is just an experiment. Which is very very positive, BTW.
Hope you like it.
Smarter (fancyer?) console output for CLI tools
You may have noticed or not, but recent changes in the core lead to enhanced console output for get CLI tool. We use utf-8 as much as possible, to include icons and special characters and colors which give a bunch of informations without cluttering your terminal with things to read.
Thanks to the recent core rewrite, we have now high-level methods and properties for building these output informations, and get has an internal cache / invalidation mechanism to avoid wasting CPU cycles.
New things coming for the TestSuite
I'm currently reading a nice TS-framework-related article, and some others (search for " nose versus py.test with google"), and I'm seriously thinking about (sort of) getting rid of, or enhancing a lot our home-made testsuite.
Sure, our testsuite has advantages of very-small-scenarii writing, very-high-level (command combinations, varying context, etc) testing, auto-included teardown commands and much-much-beautiful output (very human friendly IMHO). But we now need low-level unit testing in Licorn®, and the TS can't handle this without a consequent rewrite.
There is currently no way to test something as simple as a method raising an exception when given bad input, and I miss this. BTW, now that our code base is increasing fast, I'm looking for a way to split core.py into smaller files, and spread the test methods in the relevant part of Licorn®. Neither to say that auto-collecting test methods would be more clever than needing to chain them in core.py.
Perhaps we will acheive some sort of combinations between the two worlds. Beiing capable to test things in the background while hand-checking a current failure is something I'm not yet ready to loose.
The new ConfigurationFile class (currently beiing written and tested, thus the research and this blog entry), based on the pygments parser is something that needs to be very-very carefully tested before use, and the current TS can't do this easily.
More to comes in the very-near future, both works are progressing alongside.
PS: additionnaly, have a look at django specific testing. Generaly speaking, This might me a worth read, too.
New Event Manager
Every part of Licorn® can now emit events, and any other part of it can setup a callback to receive the event arguments, to do what is needed. Events callbacks can be run synchronously (with method L_event_run()) or not (with method L_event_dispatch()). In this case, there is no guarantee of callback order call, jobs can be parallel because they are handled by the service facility. Example:
Emitter side:
from licorn.daemon import priorities, InternalEvent
# L_event_dispatch is a builtin, defined by the daemon at start
# no need to import anything, to use it.
L_event_dispatch(priorities.NORMAL,
InternalEvent('main_configuration_file_changed'))
Receiver side:
[...]
def main_configuration_file_changed_callback():
""" this method will be auto-collected if the surrounding object
is a controller, or a CoreUnitObject whose controller defines
self.__look_deeper_for_callbacks (typically ModulesManager
does it). """
#
# do whatever needed with LMC.configuration, whose
# contents already reflect the change.
The EventManager is ready to operate even before the INotifier, and just after the LMC is setup. It will collect all callbacks of the controllers, backend and extensions on start.
It is stopped last, at the end of the daemon life.
Note: the InternalEvent instance can be given a callback argument (any callable() can apply for the job), to resync the Emitter at the end of event callback execution.
LTRACE enhancement in the daemon
Starting from now, the only thing you've got to do to enable LTRACE is to set the environment variable. If the daemon finds it at start, it will refork itself with the appropriate options to enable it.
The LTRACE status is maintained while restarting from inside the daemon (this was not as easy as it seems to implement).
Bonus: just export LTRACE=std to get a nice but not so verbose LTRACE output, you will notice that the only places where the daemon hands on stop/restart are justified (__unihibit_udisks() and PyroFinder queue not empty). In this case, just wait. The udisks thing can take up to 10 secs, and the PyroFinder one, up to 20 secs (i'm searching how to lower this one, the Pyro timeout is already set to 5 secs).
User checks customization
Thanks to Robin who did all the hard work, chk can now be customized for users. This allow to avoid ACLs for a certain file hierarchy, or force some custom ones on another. For instance, I use these, in my ~/.licorn/check.conf:
source NOACL build NOACL Projects NOACL
This allow debian tools not to crash when building packages or scanning source trees.
There is a little loss in performances, but the win is clear compared to before. We will optimize the code on next performance run (there is a dedicated ticket for that).
System ACLs rules can be fully customized (with only a few sane exceptions) by administrators. For more details, there's a bunch of related documentation!
Better trace customization
Starting from now, ltrace understands binary exclusions. You can debug the daemon and CLI tools with better cherry-picking of trace messages:
export LICORN_TRACE='core^configuration^groups' sudo python /usr/sbin/licornd -vD
This will trace everything in the core, except configuration and groups messages.
Binary inclusions still work, and you can combine the two:
export LICORN_TRACE='foundations^base^objects|core^configuration^groups' sudo python /usr/sbin/licornd -vD
Which will debug same level of core, but added foundations, without base and objects modules (which are very verbose and very low-level).
NOTE: the order matters when you write components name in the variable:
- you can write 'core^users^configuration', it will work because users and configuration are subparts of core.
- 'configuration^users' will display configuration messages only (users is not a subpart of configuration, nothing more is excluded).
- 'foundations|core^base^users' will work as expected.
- 'foundations^users|core^base' will not substract users from core, because core is processed later by the ltrace parser. base will be substracted from foundations, though.
Removed silly limitations on system groups with LDAP backend enabled
Responding to changeset 279 and explaining the reasons of changeset 304, there is now no more limitations when creating system groups.
If the LDAP backend is enabled, the newly created system group will go into the LDAP backend.
This makes NFS works perfectly on LDAP-enabled clients, which now see all groups when listings ACLs and standard posix perms.
Now, the licorn-ldap-server debian package tries to move every important pre-existing system group (acl, licorn-wmi) from the Unix backend to the LDAP one. This will implicitly install them network-wide and avoid the need to create them on every ALT®. Finally this will help propagate admin privileges on every client.
New set of commands for privileges
If you know about privileges, you know you can manipulate them only via the configuration object, like this:
get config privs sudo add group licorn-wmi --system sudo mod config --add-privileges licorn-wmi sudo add group remote-ssh --system sudo mod config --add-privileges remotessh sudo mod config --del-privileges licorn-wmi,remotessh get config privileges
Now you can handle them kind of "directly", like this:
get privs sudo add group licorn-wmi --system sudo add priv licorn-wmi sudo add group remote-ssh --system sudo add priv remotessh sudo del privs licorn-wmi,remotessh get privileges
Which is quite simpler, and - I think - more logical or consistent with the rest of the command set. The code lies in changeset 310.
