Unbelievable amount of spam

I was surprised that Blogger didn't implement trackbacks. I think to remember to have read somewhere that trackback spam was one of the reasons. And if my experience with News.Profoss is an indication, it might be a valid reason.

Yesterday I had deleted all Trackback spam on News.Profoss, and I was surprised this morning to see, in the admin interface, 12 pages with about 50 trackback spams each. That's 600 spam in one day! On a recent and quite modest website.

I better understand now the decision to not implement Trackbacks. There are spam detection techniques for trackbacks, and I installed it on News.Profoss, but this still takes resources and bandwidth. On a platform hosting thousands of blogs, it makes for a significant resource spilling.

Even some other blogging services like Wordpress implement and propose Trackback spam filtering, I can understand the decision to not propose it due to the spam problem. I'm not even sure I'll leave trackbacks available on News.Profoss. I'm not sure it's worth the pain.

Free Software graphics problems illustrated

With Adobe's John Loiacono made some noise with its post title "Innovate or Integrate?". And although publishing Adobe products under an open source license wouldn't prevent them to integrate and innovate, I understand that Free Software graphics solutions cost more when used intensively than the license cost of Adobe's products.

I've already blogged about my problems with FOSS graphics solutions, and here's another illustration. I need to provide a 120x120px logo of the Profoss. As I have material in SVG, I generated a bitmap from Inkscape. This is a png of good quality:

The problem is that I have to send a GIF. Krita in Ubuntu doesn't support GIF. ImageMagick(s convert command is easy to use:

convert profoss-logo-120.png profoss-logo-120.gif

and gives this result:


Disappointing isn't it? The URL is nearly unreadable, and doesn't look professional.

After looking at IMageMagick's documentation, I tried
convert -background White -flatten -channel A -threshold 65% logo-profoss-120x120.png logo-profoss-120x120.gif
gives a better result:


Let's try with the Gimp (indexed to default settings):
Still not good...


I tried to specify the color index in Gimp, to change the color of the text, to no avail. Finally I asked to a friend to generated a gif, and he used Adobe's ImageReady, and here's the result:


It's much better, isn't it? And I specifically asked if settings had to be adapted to get this result. The answer: "Just an export".

The funny thing is that he wanted to use Fireworks to do the export, but it refused to start (for an unknown reason, it worked fine before!) , so he had to fall back on ImageReady. Clearly shows even proprietary graphics software has its quirks :-)

Free and legal music streaming

Deezer lets you listen on your PC, for free. It is streamed like with last.fm, but you can actually choose which song you want to listen to. Deezer has reached an agreement with the SACEM, the musicians association in France. Some problems are looming though....

Deezer is the new iteration of blogmuzik.net, which was claused earlier by... the SACEM. It seems to be launched by Free, the french ISP.

The interface is well done, but the site is quite slow at the moment. Worth a try I think.

Add Yahoo! Pipes to your toolbox!

TodayI added a feed to the Profoss website. The problem is that the Profoss website contains news in different sections: one for each event, and a general news section about the Profoss itself. And I didn't want to publish multiple feeds: each time a new event is announced, people would have to subscribe to an additional feed; and each time an event is over, people would have to unsubscribe as no more news would be published.

I could have developed this on the server, but it was much easier to generate a feed for each of the news sections, and combine them with Yahoo! Pipes. I had never used Yahoo! Pipes before, but this didn't prevent me of getting a working (and definitive) solution in less than 10 minutes! I created 3 feed reader elements (one for the Asterisk event, one for the virtualisation event, and one for the general news) I then added an union node that combines all input in one output, and connected the feed readers' outputs to the input of this node. As I wanted the items to be ordered by publishing date, I added a sorting node, and specified the publishing date as sorting criteria. After connecting the output of the sorting node to the pipe's output, I was done. Running the piped provides a feed combining all items of the individual feeds ordered by publishig date.

Using the url of the rss output of the pipe with FeedBurner gives the final feed to subscribe to.


This is a very simple example of Yahoo! Pipes use, but the fact that it took me only 10 minutes to get the definitive solution working is a good indication it is very intuitive and effective. It is well thought out, eg you can see the output generated by each node of the graph, which helps you debugging your pipes. There's also good documentation, and the great variety of nodes available (there's a location extractor node for example!) should help you manipulate feeds in a lot of situations.

Using it made me think of CloverETL, also a great tool, but for data(base) migration and manipulation.

Building a webcam recorder with VLC, Ion and Ruby

I promised to build a system to let people leave video messages at a party. The requirements were:

• people have to see the stream while it is recorded, so they're sure they are in the video
• the system has to be extremely simple to use, possibly requiring the use of only one keyboard keypress to start and stop recording
• the recorded video have a maximum length, so that we avoid recording when someone forgot to stop the recording after his/her message

After looking at several options, including GStreamer and Ffmpeg, there was no easy solution in sight. And then I took a look at Videolan, also known as VLC. Here's how I got to a working solution. Note however this is a quick hack.

With VLC's WxWidgets interface, it is easy to open a video4linux device (I'm using QuickCam Express II): File > Open Capture Device:



In this dialog, check that the correct device is opened (I open /dev/video0), and things should work as expected: when you click "OK", you see the images captured by your webcam.


In the dialog above, you might have seen the checkbox "Stream/Save". Checking it enables to save the stream from your webcam to a file, the spec of this file being set in the dialog appearing when you click on the Settings Button:


From top to bottom:

1. Check "Play locally", as this shows the stream captured by the webcam on your screen
2. check "File", as this will save the stream coming from the webcam into the filename you enter in the textbox.
3. choose MP4 as "Encapsulated Method". You need to choose this carefully as not all encapsulation methods accept all format. Check the VLC streaming features page for more info.
4. According to the Encapsulation Method you have chosen, choose the right video and audio codec. For MP4 it's respectively mp4v and mpga. If you only want to dump the video, only the video checkbox has to be ticked.

Clicking ok in each dialog will show you the stream in VLC's window, and dump it in the file you entered in the configuration dialog.

Now that it is working with the interface, the next step is to get it working from the command line. As you may have noticed, when you set options in the configuration dialogs above, a text field is filled accordingly. For example, when setting the configuration in the "Save/Stream" options dialog, the top text field has its value set to
<br />:sout=#transcode{vcodec=mp4v,vb=1024,scale=1,acodec=mpga,ab=192,channels=2}:duplicate{dst=display,dst=std{access=file,mux=mp4,dst="/tmp/test.mpg"}}<br />
which contain all configuration option set in the dialog.

And in the "Open Capture Device", the bottom text field has its value set to

v4l:// :v4l-vdev="/dev/video0" :v4l-adev="/dev/dsp" :v4l-norm=3 :v4l-frequency=-1

These configuration strings can be used on the command line when launching vlc. Each v4l corresponds to a long option passed to vlc. :v4l-vdev="/dev/video0" becomes --v4l-vdev /dev/video0 on the command line.

We end up with this command:
<br />vlc --v4l-vdev "/dev/video0" --v4l-norm 3 --sout '#transcode{vcodec=mp4v,vb=1024,scale=1,acodec=mpga,ab=192,channels=2}:duplicate{dst=display,dst=std{access=file,mux=mp4,dst="/tmp/test.mpg"}}<br />


The next step is to start and stop the recording from a script. This is possible thanks to the RC interface offered by vlc. the RC interface, launched by passing the option -I RC to vlc, gives you a command prompt to control VLC. With the option "--rc-host localhost:4444", this command interface is even reachable by telnet on port 4444.

Typing help at the command prompt will give all commands available, but we'll only need:

• add v4l:// to start the recording. It opens the v4l device passed in the options of the command line, and outputs the stream to what we configured on the command line as well: the file and the screen.
• stop to end the recording

Before building an interface on this, I had to correct a little problem: the image displayed was way too dark. In the vlc interface, in Settings > Extended GUI , you have access to Image adjustments, where you can activate and set Hue, Contrast, Brightness, Saturation and Gamma correction. These can be set and activated from the command line with the corresponding option. Here are the options I had to use:

--contrast 1.9 --brightness 1.7  --saturation 2.3

I've had problem sometimes for the options to be taken into account. Going through the vlc extended GUI and enabling the image adjustements usually fixed it.

Building a little interface on this is quite straight-forward. I used Ruby and Tk. The GUI is made of.... 1 button used to start and stop the recording. It is triggered by pressing the "Enter" key. The control GUI also shows how long, in seconds, the person still has before reaching the maximum lenght of the recording.



The script starts VLC with the RC interface and the telnet connections accepted on port 4444. All commands are passed to VLC through the telnet connection. VLC always stores the dumped stream in the same location, and at the end of the recording session, the script moves it over to a definitive location, with a unique name based on the timestamp.

The problem with this setup in a "standard" window manager, is that recording starts, a window is opened to display the image coming in from the camera. this window covers the command unterface, which makes it impossible for the user to simply press on Enter to stop the recording, and the focus is on the vlc output window.

For Ion 3 users, there a quick solution here: split your workspace in two, and make the GUI appear in the left frame, and the vlc output window in the right. What's even cooler is that when the vlc output window is opened when the user starts the recording, the focus stays with the command GUI.

I will assume that you have installed Ion3, and copied the standard config files to ~/.ion3, as explained in the Ion3 documentation.

I created a new user, for which I set Ion3 as the window manager to use. I'm using kdm, so I edit .xsession with this content

exec /usr/local/ion3/bin/ion3

and at log in I choose the "Default" window manager.

When you log in, you get a workspace with one frame. To split it vertically, use Meta+K S. You can bind the key you want to Meta in ~/.ion3/cfg_ion.lua. I set it to Meta4, which is the Windows key on my laptop.

To identify the frame, it is necessary to give them a name. Focus the frame you want to rename, and press Meta+F3. This brings up the Lua code interpreter. Type the following command:

mod_query.query_renameframe(_)

and press "Enter". You are then promted for a name. Give the name app to the left frame, and vlc to the right one.
Ion automatically saves your workspaces when you log out. As the app is running under a specific user, this will not have to be repeated.

To make a window appear in a specific frame is achieved with the defwinprop directive in the user configuration of Ion. To identify the properties of a window, the xprop command can be used, or, even better when using the Ion WM, use the frame context menu (left click on the title bar of the frame, of Meta+M) to get the "Window Info":



Once you have this info, you can configure the GUI window to appear in the left frame.

The class of the command GUI is Webcamrecorder.rb, the instance is webcamrecorder.rb. It should go in the app frame, and focus should go to this window. This gives this configuration line in cfg_user.lua:
<br />defwinprop { class ="Webcamrecorder.rb" , instance = "webcamrecorder.rb" , target = "app", jumpto = true}<br />


The problem with the vlc output window is that it has no class nor instance property set... Only the title is set to "VLC XVideo ouput", which is an identification information not supported by defwinprop.... I set this in the cfg_user.lua file, so that all windows except the control GUI appear in the right frame:

defwinprop { target = "vlc"}

I set the last line of the cfg_user.lua file to

exec('xterm -e /usr/local/bin/webcamrecorder.rb')

to start the webcamrecorder immediately when logging in.

Note that I first wanted to place this in the .xsession file, but it didn't work. Apparently, webcamrecorder.rb has to be started from a terminal, or it won't work. I suspect this is due to the RC interface we request from vlc.

Once you have your solution up, you can clean your Ion bindings to avoid users switching frames, creating workspaces or starting terminals. I also cleaned up cfg_ion.lua to limit the number of modules loaded. I end up with these config files: cfg_ion.lua, cfg_ionws.lua, cfg_menu.lua, cfg_query.lua. Together with the webcamrecorder.rb script, it's all what makes up this quick hack.

Update: Richard took the script and made a GTK version. He posted it in the comments, together with information on how he's using v4l2. I put the script in a file webcamrecordergtk.rb that you can download. Thanks Richard!

Ordering photo prints online on GNU/Linux

I wanted to order photo prints from an online shop. The problem is that those shops upload tools are not very effective under Linux. Although they propose native tools under Windows and sometimes Mac, the solution for Linux users is a simple HTML form. When you have to upload 100+ photos, it's not very convenient..... Sometimes you get a useless, buggy, Java applet.

The best solution I've found is to use Picasa for Linux. After you import your pics in Picasa, you click order prints, choose your prefered shop, and your photos are automatically transfered. You then just have to confirm your order and you're done!

Although running on Wine, and as such not a native application, Picasa for Linux is quite good. It won't replace Digikam for me, but it's a good complement. A negative thing is that when you import pictures, it copies it under its own directory ~/.picasa, which can become a waste of storage space.

Although my prefered option would be to use a Firefox extension to upload photos to the print shop, like Yahoo Photos had, this Picasa will do well enough.

Two-pass algorithm illustrated

Here's joke illustrating what a two-pass algorithm is, as found in "The Art of Computer Programming":

Old lady on the bus: "Young boy, can you tell me how to get off at Pasadena Street?"
Young boy: "Just watch me, and get off two stops before I do"

Virtualisation is hot

As VMWare's IPO showed, virutalisation is a very hot subject. And Free and Open Source based solutions are showing good progress too. This should ensure an event you won't want to miss will take place in Brussels in january :-)

Continued learning online

Google Code for Educators has some course material as well as videos, with for example Ajax tutorials and Distributed Systems tutorials. This last contains an Introduction to MapReduce, which I already mentioned on this blog.

With YUI's Theater and Google's Googleplex videos, there's really interesting online material to keep you up to date, and learn new stuff.

Back to MapReduce: I again (second time only though ;-) read the paper "Why functional programming matters", and it motivates me, again, to learn a functional language, and Lisp to begin with. I've read Practical Common Lisp during last year's holydays, and the watch Googleplex' Lisp video, but never got to practice. And it won't be for this year though, as I first want to progress in my reading of "The Art of Computer Programming".

It's so frustrating: there's so much I want to learn! It's hard to make a choice. The good thing though is that I'm not bored :-)

Toolinux à propos du Profoss

Toolinux a publié un article sur le Profoss, qualifié d'"événement de taille dédié à la voix sur IP". Il mentionne également le site d'actualité news.profoss.eu lancé il y a peu.

Exim on EC2

When you configure your Exim on an EC2 server, remember to set the hostname or it'll use Amazon's internal, private unresolvable name. Here's what I did for the Profoss newsletter server configuration in exim.conf:

primary_hostname = newsletter.profoss.eu

This resolves this kind of error messages in you mainlog file:

Recipient address rejected: No IP address found for HELO/EHLO domU-12-31-35-00-39-A1.z-2.compute-1.internal - try again later

Discovering the Symbian platform

As I'm considering the purchase of a Symbian powered phone, I took a look at the platform, and there are some good stuff: Ruby and Ogg are available.

Other things are not that good though, especially C++ development tools that are only available for windows, although they're based on Eclipse......

Better than image embedded in word attached to mail

Sow I was looking further at Symbian development tools, and discovered their Open C lib, which "provide a Linux-like, vendor-neutral C-programming interface to core functions on S60 devices". And I wanted to watch their screen cast, and was surprised it was proposed as a downloadable.... .exe file!

Why?!

Firefox: the next step

Remember the time when many doubted Firefox would reach 10% market share? It's been some time now that Firefox reached and surpassed this level, and the evolution of the project makes it look to the next step: improve rentention rates (this is the percentage of people downloading Firefox actually continuing to use it). This retention rate is suprising low, at about 25%, and a new plan is in the works to improve this retention rate.

I was actually surprised at this low retention rate. I thought people downloading Firefox were already convinced that it was needed to switch, which would have given a higher retention rate than 25%(only 1 in 4!).

I haven't seen documentation about their methodology to get this 25% figure. It would be an interesting read.

[SOLVED] PDA Keyboard too small

You do have a hard time trying to hit the right key on this damn small keyboard of your PDA? Here's, the solution: get your thumbs whittled. After aesthetic surgery, is this the start of geek surgery?

Open source VoIP doubts?

Reading the article entitled "Doubts persist about open source IP telephony", I'm happy to see that the first edition of Profoss is targetting the exact questions mentioned in this article: long term costs, comparison with proprietary solutions, security, deployments in professional environments plus integration with your current system, Free or propietary.

If you're looking for objective information about professional use of Asterisk, I hope to see you on 9&10 october in Brussels! I'm sure you won't regret your participation!

Vim in Eclipse

If you're a vim addict, but need to use Eclipse, you can have your way thanks to the Vim Plugin for Eclipse.

It is based Vim tip #1031 to embed vim in your Java applications, thanks to the Java terminal emulation JTA.

Netbeans alsa has an external editor module, but things don't seem to be active, with no Vim 7 support.

Update: on the other hand, Netbeans has nbvi, a Netbeans module to integrate a java vi/vim clone. And this module is actively supported, with a release early july (2007).

Freespire 2.0 released with proprietary bits

So, Freespire 2.0 has been released by Linspire. It is based on Ubuntu and "Enhances the Popular Ubuntu Distribution by Adding Proprietary Software, Drivers and Codecs". This will be shocking some people, but is seen by others as very positive.

I've personally never used Linspire nor Freespire, but their focus on non geek PC users and their support of several FOSS initiatives are certainly positive. Is that erased by their inclusion of proprietary software in their distribution? Is it that negative if this inclusion of proprietary software enables its distribution to be used by non geek users, and this way introduce Free and Open Source software to new users?

There could be another way: educating people. But is it really possible to educate people so that they understand the importance of open formats and free software? To the point that they refuse to use proprietary software? I'm afraid not. A lot, and mean a lot, of people really don't care and don't understand the importance of Free Software and open formats.

These days, even some Free Software diehards are communicating with MSN for practical matters. Maybe we should accept Freespire's proprietary bits that give Linux users a legal way to watch proprietary video formats for practical matters? At least Freespire's users will be watching these videos legally, not like some Linux users using windows codec dlls with (e.g.) mplayer.

Anyway, whatever the way we get there, I hope we can get rid of those proprietary formats. When you see how things have progressed with the Open Document Format, and the existing alternatives like Ogg Vorbis and Theora, there's still hope. In the mean time, some people will compromise but hopefully they will not forget the importance of open formats and Free Software.

Linux mobile phone from Motorola

Motorola is shipping a linux powered phone.... in the US. The RAZR2 V8 is apparently focused on multimedia capabilities. No trace of it on the Motorola Belgium website.

L'arroseur arrosé

Un journaliste travaillant "undercover" espérait pouvoir enregistrer des présentations compromettante durant une réunion de spécialiste de la sécurité informatique. Mal lui en a pris: les organisateur ayant été mis au courant, ils ont organisé un nouveau jeu: "Spot the undercover journalist". Résultat: une horde de participant poursuivant la journaliste pour la filmer et la photographier.

Google mashup editor in limited beta

The Google Mashup Editor is available in limited beta. Reading the introduction (I'm not part of the beta :-)), it seems to be a cool product which facilitates development of mashups with eg easy access to feeds and their structure and Google's infrastructures such as maps. It's all built with GWT. I'm curious to test it, and see how this compares with yahoo's pipes

Yahoo gets behind Hadoop

Hadoop, the (recent) Apache implementation of MapReduce in Java, has seen its developer Doug Cutting hired by Yahoo.

The funny thing is that the first time I read about MapReduce was in a (very interesting) paper titled "Why functional programming matters" which was written in .... 1984!

It's not always the lates technology that brings advancement: good use of existing technology can do a lot too. Ever heard of Ajax? ;-)

Amazon Flexible Payment Beta available

Amazon has started a limited beta of Flexible Payment System that they describe as "the first payments service designed from the ground up specifically for developers".

Features look interesting, and try to make micropayments possible (I remember having read somewhere that Amazon was seen as the only company that would be able to make micropayments viable due to the amount of credit card transfers they do monthly): for example you can aggregate several micro-payments in one transaction.

It also features easy billing of Amazon shoppers as the same credential can be used to log in Amazon and pay with FPS. FPS also uses the data stored in the shopper's account, so no need to ask credit card numbers.

It seems flexibility is key to their offer. Apparently it would be possible to build a market place where you aren't the paid entity. Their example is proposing mp3 downloads where the payment goes to the artist, with the market place getting a share of the transaction.

But what about the fees? Well, here's the situation for credit cards: For a payment higher than 10$, Amazon charge 2.9% + 0.30$, same as Paypal. But Paypal lowers the fee for transaction higher than 3000$. And Amazon changes the fee for payments lower than 10$: 5.0% + $0.05. Google checkout is free for the moment, but will charge 2%+0.20$ from january 2008.
When you compare these solutions, it becomes clear that Amazon is looking to grab market share in smaller payments: the smaller the payment, the cheaper they are compared to their competitors, the biggest the payment, le least competitive they are (except if you qualify for their volume discounts, which give you the same rates as Paypal):

Amount	Amazon fee	Paypal fee	Google fee
1	0,1	0,33	0,22
5	0,3	0,45	0,3
11	0,62	0,62	0,42
3001	87,33	75,33	60,22

Now for the bad news: this beta service is reserved to developers having a credit card issued by an american bank.... As Google Checkout is only available to US and UK vendors, this leaves me with only Paypal as a viable solution. It's working fine for me in Myowndb and Profoss, but although the documentation is complete, using their development sandbox is time consuming, the forum is not great (just did a search that ended with a "ODBC Drivers error '80040e31'") and the lack of competition made them leave useful features unimplemented (eg reactivate a subscription that expired).

FPS is the latest Amazon webservice released. They started in 2002 and have built business on each of them, as Jeff describes.

Remote Device Acces by Nokia

Nokia now proposes a free service to Forum Nokia members: you can test your application on different Nokia devices through the internet, and it's called Remote Device Access.

The E90 isn't there yet, but it could be a good way to see how it behaves with different applications available before I possibly get one. The cool thing is that it's not an emulator, as the name implies. What you see is the display of a real device connected to Nokia's server. You reserve the device you want to test for a timeslot, and you can play with it. Here's a blog post describing how it works.

Extjs 1.1 released!

Wow, first Yui 2.3.0 and now Extjs 1.1!

This is the first stable released not mandating the use of a third party library (YUI, prototype or jquery) which gives a way to have a smaller size for the base library to download in a web page to use Ext's power.

The HTML editor is also a big addition to the library.

I've already been using 1.1 beta in production, and it was really solid and enabled me to build really advanced features very rapidly. And as I'm not a designer, Extjs' good look by default is really a strong point for me (Have you seen the layouts possibilities available?).

If you haven't made your choice on which Ajax library to use, take a look at Extjs! You get flexible data sources (reading XML, JSON) that you can use in different widgets (grid, combo box), layout facilities to build desktop-like apps, localisation, inline editing grids and lots of other widgets, a fast CSS selector, great documentation, active community to ask and reply to questions, user developed extensions, etc....

Congrats to Jack Slocum and all contributors for this excellent release. Looking forward to what Ext 2.0 will bring us!

YUI 2.3.0 released

Version 2.3.0 of Yahoo's User Interface library is available. Although I'm using Extjs, which can now be used independently from any other library, I still keep an eye on this library, and it's good to see it progress and add interesting stuff (rich text editor, a loader utility, an image loader, unit testing, etc). The documentation available with YUI is impressive and lets developers get up to speed rapidly.