Random thoughts and experiences with Linux

More thinking on software bundles for Linux

2011-11-01T17:18:00.000-07:00

The post Rethinking the Linux distibution made me revisit some of the ideas which I had in the past trying to address which in my opinion are major limitations in the current main packaging systems:

No support for multiple versions of the same software
No support for rollbacks

A software bundle composed of the application and all it's "non core" dependencies can also add some other benefits:

Cross Linux distribution delivery
Fine grain control of libraries and options used by an application
Reduced complexity with the removal of dependencies management

The disadvantages are:

Increase on Disk and RAM usage from software containing different versions of common libraries
Applying security fixes on dependencies requires re-creating/re-distributing every affected bundle

Possible approach for implementation:

Compiling

Adapt an existing source base build system like Arch's "makepkg", with the following changes:

Run time prefix must be set to /opt/bbundle
Build definitions for dependencies must be contained in the master build definition (this will lead to build definitions redundancy across bundles but will remove the risk of breaking builds by sharing dependencies building rules)

Bundling

The bundle file format should be a commonly used archive format, since tar does not provide indexing, .zip is a better option. Having an indexed archive will allow to reduce download sizes by inspecting the bundle contents prior to the download and skipping the download for common files found at installed bundles. In order to save on-disk space, the bundle installer should check for identical files across bundles and use hard links instead of duplicating files.

Installing

Bundles installation should be as simply as extracting the bundle archive into /usr/local/bbundle/bundle_name. A watching service must identify .desktop and other exportable resources and make them available from the host desktop environment.

Life changes

2011-05-12T06:40:00.000-07:00

It has been more than 1 month since I left the Ubuntu community and started exploring other Linux distros. Meanwhile the increasing personal concerns added to the my country's economical situation prompted to re-evaluate where I am investing my life time.

In short, I need to engage in profitable activities otherwise I may fail to support my family.

While I will always be a strong FOSS supporter, because I believe on it's values. I no longer have the time for significant involvement in non profitable projects.
I will continue using and consequentially involved in the Linux ecosystem because it allows me to be more productive, both on my job and on other projects I may get involved with.

Building RPMs vs Building DEBs

2011-03-14T08:01:00.000-07:00

Having a large experience with Debian package building it's refreshing to try something else. Last week I have learned the basics of RPM package building.
Here are the differences I have found so far and my opinions about them.

The RPM .spec file package contains both package metadata (description, dependencies, etc) and compile rules while on DEB you have the data split into different files. I remember that on the beginning it was hard to understand the purpose of all those debian/* files, I have found .spec files easier to understand.

The RPM .specs allows conditional building, during build the build target release can be used to dynamically adjust build flags, dependencies etc. While you can achieve this on Debian using some auto generation mechanism (debian/control.in), it is not naturally integrated in the building system, debian/* contains metadata and rules for a specific target system.

Not so important but a nice feature it's the support for translated description/summary on RPM packages.

Some differences between Fedora and Ubuntu

2011-03-05T03:00:00.000-08:00

I have already noted a few technical differences between Fedora and Ubuntu which I am going to comment.

/tmp cleanup
Fedora does not automatically remove /tmp contents on reboot, I prefer Ubuntu's behavior, applications do not rely on tmp contents across reboots and regular users should not be working on /tmp. If there is no other automated cleanup mechanism -did not check yet-, on the long run the user will get a full root file system.

Repository information cache
I don’t have a YUM technical background, so please excuse me if I will write something terribly wrong here.
From an user perspective I have noted that yum does not have an explicit cache mechanism, you don’t need to explicitly update the cache. The good side it automatically gets the required information when a new repository is added and frees the user from a repetitive action. The bad side is that it may introduce some network/time overhead during package management opertions.

Software update policy
I did not read Fedora’s update policy yet but I have noted that they provide regular release upgrades for some software, piding was updated to 2.7.10 from the regular updates repository.

Impressions from Linux Mint 10 Main Edition

2011-03-04T15:27:00.000-08:00

Now that I am confident on the Fedora install I could afford to use my other partition and try Linux Mint 10 as suggested by a friend.

The out-of-the box visual experience was one of the best I had so far with a Linux distribution, the menu -mintmenu- seems highly inspired in MS Windows menus, from my reading it's a fork of the SLAB Menu, I really loved the easy navigation and search capability.
Linux Mint 10 is based and compatible with Ubuntu 10.10, the default configuration points to the linux mint repository plus the Ubuntu archives for the usual packages.

Besides the menu they provide their own set of tools, you can easily identify them with ls /usr/bin/mint*, some of them are just wrappers or tiny tools, but it shows they are working not only on cosmetics.

I did not find any documentation about their decision making process or governance in general, but I have sent an email asking for information.

They do care about community feedback, judging from the running poll .
There is a Debian based version (which I did not test yet), and it seems they will be deciding about switching to Debian for other flavors.

I will keep an eye on it, it may be a nice project to join.

First day with Fedora 14

2011-03-04T00:58:00.000-08:00

I am starting my adventure searching for a new FOOS project to get involved with. I still think that a Linux distribution is one of the most interesting type of projects. We have so much Free Software that is not reaching yet those who would benefit from it.

Yesterday I have switched from Ubuntu 10.10 to Fedora 14 I have chosen to try Fedora because it has an open governance model with a clear leadership. While there are a clear special capacities from the sponsor (RedHat), at the highest level the project is managed by an Executive Board, the board is composed with a mix of RH appointed and community elected members.

Now back to my first day experience, I was afraid that it would be an hard experience, I use Linux for my primary workstation so it really needs to work.
The install was smoothly, I did found a bug on a specific case of setting up an user with an existing home dir containing invalid symbolic links, nothing serious.

I was able to install all my job required tools using Fedora 64 bits thanks to the multiarch supports which allows to install both 32 and 64 package versions, this is something I could not achieve with Ubuntu 64 bits, ia32libs is not sufficient for my case.
All the software that I needed was available from repositories or as an .rpm from upstream: Filezilla, KeepassX, Zim, X-Chat, Skype, VirtualBox, gnomedo, tsocks, eclipse, cairo-dock, geany. dropbox, pidgin .

During a full work day I have found no significant usability differences between Fedora an Ubuntu.

The only issue I had so far was related yum repository errors, this is one of my next steps, to understand Fedora repository types and package building.

I will also try to understand if the distribution itself is effectively governed as documented.

Stepping down considerately

2011-03-02T13:36:00.000-08:00

This is the right time to apply the last guide line from Ubuntu's Code of Conduct.

I have started as an Ubuntu user in 2005, I have found it a promising project mostly because it was aimed at "humans" users, while most similar projects had still a greater focus on developers or development oriented aspects.
Getting involved was easy, the developers could be found on IRC some of them more friendly than others but always there, a point of connection with the community.
As soon I had some know-how I have started participating in the forums, each question was an opportunity for teaching, learning or improving, it was a great experience.

During this stage I have found that a lot of answers were related on how to get a specific version of a software, or people failing to do it. The most frequently answer was teaching how to build from source. That did not seem good to me.
We were promoting all this great thing of Free Software, but we were unable to deliver the latest version meeting a particular need without requesting the "human" user to get some application building skills ?

I have tried to engage the packaging (MOTU) team, It just didn't work for me. I was too eager to cover this need -only partially addressed by the back-ports project-. I did not find the process appealing I had no idea how to improve it, I just had minimal packaging skills.
People were not asking for the proper package, they just needed one that worked without disrupting their system (like they frequently did, compiling or installing from other releases). The GetDeb project was setup, it delivers packages to thousands of users.

Lately I have mostly participated on AskUbuntu.com which in my opinion is Ubuntu's mostly valuable free support center.

I will be looking for other Free Software collaboration opportunities, with a strong leadership that practices open governance and uses decision capacities that comes from transparency, straight discussion and communication.

Thanks to Mark Shuttleworth for setting up a great project and paying the salaries to so many brilliant people.
Thanks to all the Ubuntu users and developers for helping and letting me help building a Free Software and Open Source solution.
.

Ubuntu Community Council Experience

2011-03-01T15:00:00.000-08:00

As reported in a previous post I have requested the Ubuntu Community Council for the definition of a control process for non technical driven changes .
Today I have attended to the CC meeting and I would like to share my experience.

I was able to expose my concerns, I got a long feedback from Mark, and as I understood his position and because no one was able to identify actions that worth be taken, everything is fine with Ubuntu change management. If you need more information about changes the recommended process is just to ask Mark he will point you to the right person.

My feeling about the CC board per si, is that, except for Mark which had Canonical/himself all over his conversation, the Ubuntu community was properly represented.
However because the CC lacks any executive capacity, that meeting felt mostly like a social event to get the pulse of the Ubuntu community, not an event to have effective discussion of ideas.

Logs available at: http://irclogs.ubuntu.com/2011/03/01/%23ubuntu-meeting.html

Free and Open Source Gaming Challenge Idea

2011-02-27T14:46:00.000-08:00

Hello,
sometimes I find myself search for gamins at playdeb, for my friends, family kids or just for my own entertainment.
I believe that there is still a lack of awareness about Free and Open Source Games and we can do something to improve it.

The following idea is to run a FOS gaming challenge, unlike the traditional network based / score based competition this challenge would be about achieving a per game predefined objective for the largest possible number of FOS games. The primary goal is to reward gaming diversity, not expertise.

The idea is presented at:
https://docs.google.com/present/view?id=dnnmb2s_59f64pmhdv

If you like it and have a facebook account, check our facebook page:

If you want to get involved in the discussion and eventual organization of the event please subscribe to the mailing list:
https://lists.sourceforge.net/lists/listinfo/fosgaming-challenge

Please share this idea with those that you believe would be interested in participating, if we can gather the resources and sufficient interest this is likely to be more than just an idea.

Thanks to the GetDeb team mates which helped refining the idea and building up the presentation.

Non technical driven changes to upstream packages

2011-02-27T14:42:00.000-08:00

Hello,
today I have sent this request to the Ubuntu Community Council which I believe to be in the interest of the Ubuntu community:

In light of the present Banshee default configuration change but also taking in account past events (bug 642839) I would like to request the definition and implementation of a control process for non technical driven changes.

As far as I understand such process is already in place for technical changes, and covers mostly stability, security and miscellaneous integration driven changes, with review/authorization being granted by the Ubuntu Technical Board when required.

In the absence of a similar review/authorization request process for non technical changes I am afraid there is an high risk of changes being introduced without proper assessment and communication.

This request does not seek in anyway to limit or condition Canonical's business authority for the Ubuntu trademark and product management, however such authority must be used in a way which is transparent to the Ubuntu community.

Software Center validating packages quality

2011-02-26T15:46:00.000-08:00

Today I have found bug 712377, it seems that Software Center is going to check packages quality and refuse to install them.

This change is likely to affect many 3rd parties, does anyone know if it is planned to be enabled on Natty and where can we find the change specification/discussion ?

GetDeb: New build server

2011-02-26T02:00:00.000-08:00

The packages building server was moved to a new infrastructure, the resources increase decreased the build time significantly.

We have also developed a minimalist report with the name and logs of the packages recently build, you can check it at http://build.getdeb.net/ .

Dear Ubuntu Community Manager

2011-02-24T14:28:00.000-08:00

Jono,
could you please provide us some insightful information about what REALLY happened regarding the Banshee music stores default configuration ?

We were firstly informed that there was a negotiation attempt between Canonical and Banshee developers and which terminated with Caconical's terms being rejected.

Now we have your communication which attributes a mishandling responsibility to Cristian Parrino, however it does not provide a clear understanding on what happened.

Is this new a plan a proposal to the Banshee core developers ? Was it accepted by both parties ?
The perception that the Ubuntu community (which Canonical is part of) followed-up up a failed negotiations by communicating an unilateral plan with different terms provides a sense of questionable intentions.

The Free Software License of Banshee grants Canonical the right to manage the announced changes without involving the Banshee core developers, how was that right used ? Did Canonical trustfully considered to establish mutually accepted terms or was this just a mishandling by setting up a negotiation which was not intended ?

Thanks in advance.

Mono and the Open Source Cannibalism

2011-02-15T05:30:00.000-08:00

The recent post from Miguel de Icaza demonstrates an incredible sense of business opportunity: Mono delivers what a mobile developer should care about, be everywhere.

Why now ? Has mono reached some significant technical milestone this week ?
Not really, but right now many mobile open source developers, supporters and business partners are concerned, let's remember them that Mono is here.

Who cares about the people involved in projects like Qt and Meego? They are just a bunch of losers, they are mobile developers so they should be using Mono anyway.

Does Canonical support help?

2011-01-04T09:17:00.000-08:00

If you have subscribed to bug 439448 you probably be wondering about the effectiveness or even usefulness of bug reporting.
It is a serious usability bug, present on a LTS release, and one year after being reported is not yet clear if it's Ubuntu specific, or the exact component causing the bug. Because it affects many, mostly non experienced users, those more than 300 comments on the bug are mostly wild guesses about causes and workarounds, digging them to find relevant information will be an expensive and futile exercise.

Do you believe that subscribing Canonical's desktop support services would help in cases like this?

If the company can win where the community fails, maybe we can setup a community economical effort to get company paid support.

Why I am still supporting Free Software?

2011-01-03T09:36:00.000-08:00

Today I was debating with a friend the relevance of Free Software, he pointed that at the current development rate is very unlikely that Linux (most common Free Software subject) will have a significant end users market share in 50 years. I remembered him that Linux powered devices already a significant mobile market share. He noted that most of the mobile users do not care or do not know that they use a Free Software powered mobile phone – I had to agreed on this.

Later today while providing some answers at askubuntu.com I have kept this debate on my head and asked myself, why I am still supporting Free Software ?

When I started getting involved in free software development, about 15 years ago, I was a young programmer wanna-be, I was eager to learn all these bunch of languages, protocols, libraries, etc. As if it wasn't good enough I could even get help and help other people, which I always loved to do. I never managed to get a job with free software/open source (and probably will never do) but despite having a good job I always felt that working and programming with free software was closest to what I love doing, knowledge sharing.

Today I no longer do programming, except for a few improvements at getdeb and some scripting at my job when I look at code this days is mostly to identify a problem or feature. For me the (open source) code has lost the magic it had a few years ago. Thankfully to my loved wife, daughter and friends I no longer have the required free time or desire to learn/work on what is required to fix bugs or develop new features. I have lost most of the capacity to use one Free Software's fundamental freedoms “The freedom to study how the program works, and change it to make it do what you wish (freedom 1)” , I can still do many things with the code, but no longer the ones I wish.

Why am I still here, an Ubuntu member, supporting Free Software yet economically dependent on and surrounded by commercial/closed source software?

I have assimilated the values of the Free Software -without the radicalism of some of it's activists-.
I believe that the ability to keep and expand such freedom is still more important than to use it.
I do not have the social skills usually required to influence or change minds, but I am sure I can reach others in ways that demonstrates the values of Free Software, which are hard to pass, specially for most people -which are not developers- .

Happy New Year 2011

Ubuntu Bug Fix Wishes for 2011

2010-12-30T14:24:00.000-08:00

I would like to see the following bugs fixed during 2011, the first randomly prevent users from logging off, the second randomly presents an “ugly” unexpected appearance.

https://bugs.launchpad.net/ubuntu/lucid/+source/gnome-panel/+bug/439448
https://bugs.launchpad.net/ubuntu/+source/gnome-settings-daemon/+bug/574296

Both are responsible for a disruptive desktop experience for many users. I also wish that the effort for the new “consistent user experience for desktop” does not keep or increase our current inability to fix such severe open source problems.

GetDeb mirror pool status

2010-11-11T15:23:00.000-08:00

I was able to identify and fix the python threading deadlocking problem, mirror-selector is fully functional now.

I have also added some statistics which allow us to check mirrors health and traffic distribution, you can check it at:
http://archive.getdeb.net/status/

Help with Python threading deadlock

2010-11-09T05:52:00.000-08:00

Hello,
some months ago I have started developing the mirror-selector daemon.
The main work is done, it is capable to run for a few hours serving some thousand requests but it then get's into a thread deadlock scenario (the web server stops responding, gdb shows all the threads waiting on a semaphore lock).
This was my first project using multi-threading with Python and I am having a bad time finding the bug. I have recently introduced some debug code that I hope will print the stacktrace for every thread and give me an hint on the cause. It is most likely related Queue management.
The mirror-selector code is not that complex, it's available from bzr "bzr branch lp:mirror-selector", if you are experienced with Python threading and can spare a few minutes reviewing the code for the possible cause, it would be helpful.

Thanks

GetDeb/PlayDeb repositories for Ubuntu 10.10 are now open

2010-10-11T13:14:00.000-07:00

We had another great Ubuntu release, the GetDeb/PlayDeb repository setup for the 10.10 is done and packages should start landing on the next days. For those which prefer to keep on the LTS release we will keep updating the packages for Lucid for at least the current release.

GetDeb: Archive traffic distribution

2010-07-23T07:46:00.000-07:00

GetDeb user base increased exponentially since it was started on 2006. The migration to a proper APT repository while providing important benefits is also a big technical challenge. The GetDeb and PlayDeb repositories are presently configured by more than 30k users, providing tens of Gigabytes from our mirror pool during traffic peaks.
On 2007 we had a single server and the traffic was unaffordable, we have gathered some mirrors and we have developed a php script which was responsible for validating files availability on the candidate mirrors and then redirect the users to them (using http redirect). This script was poorly developed but sufficient for a long time.
Before moving to APT the file requests were human originated from web clicks, now this scrip is massively used by the automatic system upgrades, it's original faults have now a much serious impact. It needs to be replaced.

I have checked existing solutions for mirror distribution:

APT mirror: method - APT supports a specific mirror: method which dynamically obtains a mirror from an URL, however it's transaction based, the same archive will be used for all requests after an initial retrieval. This means that on the beginning of transaction it should get the url of a mirror which provides all the files required by the subsequent operations. For GetDeb this is a major limitation, since we have very frequent updates (somtimes hourly) most of the mirrors would be unavailable for mirror selection because they would be out of synch, even if they do have the packages for that specific transaction there is no way to know in advance. This issue is not present with http redirects, we always return the packages index from the master server, files will be obtained from individual mirrors as long they match the master server version, regardless of the overall mirror status.

Mirrorbrain - Mirrorbain is used by mainstream solutiosn like OpenSuse's build service and OpenOffice so it was a strong candidate. After some research I have found that it detects file availability by using a database which must be kept current using a mirror scan tool which does a full mirror scan (file info: size, last modified). While this maybe great for most scenarios I don't think it is as efficient as doing on demand mirror check, our slowest mirror took >10m for a full scan, we would need large intervals increasing the risk of redirection to a failed mirror.

mirror-selector - Because I have a strong believe on the technical merit of the on demand scan I have decided to implement a mirror selection system from scratch using Python.
The utility/project name is "mirror-selector" it runs as a standalone HTTP Server whose only purpose is to handle static file GET requests, check the availability from a local directory (it must be run on a local mirror) and then redirect to an available mirror after checking that an exact copy of the file is available remotely.

The http server uses a fixed size thread pool, each web client request is handled on it's own http server thread. When mirror-selector starts a thread is started for each mirror, each mirror thread provides an input queue which maybe used by any http server thread. With this architecture all requests related to a unique mirror are handled on a single thread, this allows to easily reuse the same TCP connection by using HTTP 1.1 Keep-Alive for multiple requests. The caching facility is also simpler to implement because it works on a per thread basis.

The code is available at launchpad: bzr branch lp:mirror-selector (check the README to test it), it should be considered as alpha.

GetDeb's/PlayDeb's main archive pool was already switched to mirror-selector, we may intermittently swap to the legacy selector as serious problems maybe still be found.

To check if it's available and some stats:
http://archive.getdeb.net/status/

Something is wrong with the software publishing plan for Ubuntu

2010-07-10T01:37:00.001-07:00

Lately I have been watching a couple of threads related to Ubuntu packaging and publishing whichs makes me feel something is wrong.

With all respect, I believe Matt was probably dreaming when he came up with the "We’ve packaged all of the free software" title. There are plenty of applications not properly archive maintained or even packaged due to the lack of human resources.

Then Jorge on this article narrowed the issue to the the centralized versus distributed model. It's a bit concerning the argument which involves "iPhone’s app store", will Ubuntu app store follow iPhone's restrictive strategy because *some* people like it ?

Opinions apart, there will be a new process for applications inclusion which is great but which I am afraid will be commercially oriented. If Ubuntu does not have the resources to handle the current community oriented package inclusion process for development releases, REVU I really don't see how it will be able to handle post release inclusions unless there is revenue involved.

Python and OpenOffice - Hello World

2010-07-06T05:24:00.000-07:00

Recently I had the need to automate data gathering into an openoffice calc heet, after some frustration playing with the OO macro language (OOBasic) I did some research on how to interact with OO from Python. There is some documentation spread over wikis and forums but not that much so I am going to share my learning here.
This is a simple hello world script which will fill the "A1" Cell from the current active sheet with "Hello World" in bold.
Please note that I will be using regular python scripts launched from the terminal and which will interact with OO using a UNO socket bridge.
First you will need to start oocalc in listenting mode (to accept UNO connections), from the terminal:

oocalc "-accept=socket,host=localhost,port=8100;urp;StarOffice.ServiceManager" -norestore -nofirststartwizard -nologo

Now just execute python and try the following code:

from uno import getComponentContext
from com.sun.star.awt.FontWeight import BOLD

# Connect to soffice using UNO
localContext = getComponentContext()
resolver = localContext.ServiceManager.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext)
context = resolver.resolve("uno:socket,host=localhost,port=%d;urp;StarOffice.ComponentContext" % 8100)

# Get desktop service
desktop = context.ServiceManager.createInstanceWithContext("com.sun.star.frame.Desktop", context)

# Get current document
document = desktop.getCurrentComponent()

# Get the active sheet
sheet = desktop.getCurrentComponent().getCurrentController().getActiveSheet()

# Get the "A1" cell reference
cell = sheet.getCellRangeByName("A1")

# Set a string
cell.setString('Hello World')

# Change to bold
cell.CharWeight = BOLD

That is all for the first lesson :)

GetDeb: Ubuntu 10.04 kick-off

2010-04-30T13:49:00.000-07:00

The GetDeb and PlayDeb repositories for Ubuntu Lucid provide packages including more than 100 applications and 100 games, this was the first time we had most of the previous release packages already available on the new release day. The credit goes to Christoph Korn which tested/fixed most of the packages, thanks also to all the others which helped.

We have recently added change logs generation to our build system, when using update-manager you will be able to check the change logs before deciding to upgrade to a getdeb package.

When "recovery" mode fails to boot

2010-04-20T02:45:00.000-07:00

*** Don't try this unless you understand what you are doing ***
The recent changes to the boot process with the integration of the graphical boot loader and logger (Plymouth) and the continuous development of upstart and associated jobs brought a significant improvement to Ubuntu's Lucid boot experience.
Because the the code is new and is rapidly changing there are some good chances of bugs which may lead to an non bootable system.
Some of the bugs found (promptly fixed, or being worked) on the development stage shown that if you have an issue related to plymouth/mountall the rescue option on the boot menu will not work.
Rescue mode as implemented now depends on plymouth/mountall being started in the first place.

To work around this you will need to be able to boot without automatically starting upstart, you can do this by appending using the init kernel option, from the boot menu, press e for edit, search for the linux line, append the init=/sbin/sulogin, press ctrl-x to boot. If you don't get a root prompt then something was wrong during kernel load/initialization, or while the root filesystem was mounted (removing the quiet option from the linux kernel line might provide some hint on the problem).

So now we are in a root shell, what's next ? On my case I am going to set the console keyboard layout: loadkeys pt .
We can't yet start services, we are running from an sulogin process and we need upstart to be able to start services. We will just backup and disable the upstart services first:

# Remount the root fs in read-write
mount -orw,remount /
# Backup the upstart jobs config
cp -a /etc/init /etc/init.orig
# Nothing will start
rm /etc/init/*
# Add 2 console virtual terminals
echo start on startup > /etc/init/sulogin8.conf
echo exec openvt -c 8 >> /etc/init/sulogin8.conf
echo start on startup > /etc/init/sulogin9.conf
echo exec openvt -c 9 >> /etc/init/sulogin9.conf

Now we can start upstart: exec /sbin/init .
The console will just hang because nothing was started, except our virtual terminals 8 and 9, now let's swtch to VT8 with with CTRL-ALT-F8.
Now list the the available services: initctl list, a bit boring, only our sulogins, let's put the regular jobs in place: cp -p /etc/init.orig/* /etc/init .
Try initctl list again, a long list of services now, which you can start with "start service".

Please note Upstart is an event driven job management system, starting a service will most likely trigger other services which depend on events emitted by the first one. My next research will be on how to avoid the chain reaction, or if not possible request such a feature :)

I hope you don't need this instructions :)

Upgrading to Lucid with Virtualbox USB support renders the system unbootable

2010-04-18T03:46:00.000-07:00

Last friday Mina Shokry joined in#Ubuntu+1 reporting an unbootable system after upgrading to Lucid. Even booting with rescue mode it was hanging without any error message, apparently it was after mounting the file systems (guessing by the "mounted filesystem" message).
After several hours trying all sort of boot options we are able to identify the problem.
The /proc/bus/usb on fstab which was previously required for USB support in VirtualBox mount was failing, however upstart/mountall (not sure where is the fault) did not provide an error and simply stopped the boot process.
The bug for the mount issue is bug 565109 .
I have also filed a bug for upstart, bug 565890, simply making it verbose when using rescue would allow to easily identify the problematic task.

If you use VB's USB support make sure you remove the /proc/bus/usb entry from your fstab.

Update Manager will display 3rd party change logs

2010-04-12T08:05:00.000-07:00

There was an old bug reported on update-manager about it's lack of support for non official repositories change logs display. This was something already requested on getdeb and it benefits most people which use PPAs, checking the change logs usually provides a good hint on the upgrade risk.
I had proposed a possible solution in the past, today I have check with Michael Vogt that the proposal was ok. I got positive feedback so this was the time to help with some code.

I don't have much experience with python-apt so the research over the weekend using winpdb (the python debuger) was fundamental. I have found the function responsible for the changelog file retrieval, get_changelog() at UpdateManager/Core/MyCache.py and added the change log URI construction based on the package origin server, later Michael fixed it using the ArchiveURI from the package.
The patch introduced a new feature so it required a Feature Freeze approval, it was granted quickly, it is a small patch providing a perceived benefit.

Please note this feature depends on the changelog files being available on the server, at ARCHIVE_URI/changelogs/... (pool structure). The repository maintainers need to implement their own changelog extraction utilities. GetDeb has already implemented it, however it will only be available once we upgrade our build server to 10.04 (there are some other building related changes blocking).
A task was also submitted to Soyuz so PPA's should support it in the future.

Crashes with Ubuntu 10.04 beta 2

2010-04-10T03:13:00.000-07:00

I have been using Ubuntu 10.04 since beta 1 and it has been quite stable until this week.
Yesterday on my laptop openoffice simply topped launching, running oowriter from the terminal did not reported any error but nothing happened. I was in a rush to finish a job so I was unable to debug the problem further so I simply rebooted.
During boot the root filesystem was found corrupted, the graphical boot just kept looping changing the message on the screen from "Checking disk blah blah" to "key X por blahblah". I was unable to do recovery, next step, reboot in recovery mode.
Still not good, now it was just looping on the text console, I was unable to read the output.
Another reboot, this time I kept pressing the key that was supposed to bring the recovery console, this time I was able to get into the recovery shell after some iterations of the message loop. I was able to run fsck on the root filesystem and successfully boot.

Today I have booted my desktop (also running Lucid), was about to do some draft html for a status page, decide to install "quanta" for it.
During the install I just got a white screen with X hard locking, well time for another reboot.
I was not sure about the quanta package install status, I have found that quanta was installed by quanta.desktop file was truncated with 0 bytes.
Later I have found that other quanta dependency packages where corrupted

I have filled bugs 559915 and 559915.

Anyone with such bad experiences lately ? I am a bit concerned about having such critical failures on 2 completely different systems.

Installing a Debian schroot with debfactory

2010-04-09T00:21:00.000-07:00

I have recently had the need to teste a package on Debian Lenny, it was a good reason to add support for Debian on the debfactory schroot_build.py script.
Durint testing I have found bug 557316, you will need to add http://ftp.debian.org/debian to /etc/apt-cacher-ng/backends_debian before proceeding.

To create and test on a Debian Lenny schroot just run:


bzr branch lp:debfactory && sudo debfactory/bin/schroot_build.py -c /home/chroot -p debian -d lenny -a i386 && schroot -c lenny.i386 -p

GetDeb/PlayDeb: Ubuntu 10.04 Lucid Lynx Repository Available

2010-04-06T10:02:00.000-07:00

The repository for Ubuntu 10.04 repository is now available.
Lucid is an important release (LTS status) and we have a significant number of packages/updates which will not be available on the official repositories, with such in mind we are now building/testing only for Lucid.

Some questions and answers about GetDeb

2010-01-05T05:44:00.000-08:00

Today I have found this blog post which raises some interesting questions about GetDeb.

There's GetDeb, but I'm not using it yet. I've got questions that the GetDeb site doesn't have answers for: who runs it? What releases do they package? Are program authors involved in any way? Are the programs modified before they land in the repository? As an issue of trust, I'm more willing to add repositories like Banshee's official PPA to Software Sources over a third-party I know nothing about.

The answers:

"who runs it" - A community of volunteers, list of individuals which can upload to the "-testing" repository is available from https://launchpad.net/~getdeb-uploaders/+members . A more restricted subset can copy packages from -testing to the general repository.

"What releases do they package?" - The general policy is to provide the current stable version, there are exceptions, we provide development releases when recommended by program authors.

"Are program authors involved in any way?" - It depends, on some cases package requests are received from the program authors, on other cases we just grab the authors source and there is no interaction.

"Are the programs modified before they land in the repository?" - In general no, on some rare cases we do apply building/integration fixes, on those cases we send the fixes to the author's also.

Please do not include debian/ in your source tree

2009-12-18T07:36:00.000-08:00

In my opinion in his previous post Robert Collins proposes a change which does not help packaging, instead it would just make things worse.

DO NOT include a packaging debian/ directory into your source tree. If for whatever reason you need to provide your application out of the official repositories just upload to a PPA using the regular process with the source+debian diff .

Including the debian directory on your source does not help anyone, however it will make life's harder for people that wants to package your software. The debian/* format is standard but it's implementation, integration and component versions between different distributions and releases are not.
There are plenty of reasons for projects/individuals to provide your software built/integrated in a different manner. Including a specific version of a package building process on the source will conflict with the regular packaging process, source+buldif diff forcing other packages to change your source by removing the /debian dir or maintaining a diff for it.

I agree with Robert that upstreams should be more involved with distros and whenever possible take care of the packaging, the blockers for that are more social than technical. If there is an understanding that the "introduce a new package" process is not acceptable/friendly with upstreams then it should be improved.

If the objective was to improve packaging collaboration then just move debian/* to a VCS, that way everyone can collaborate: upstream, distribution maintainers and 3rd parties.

RSS feeds available for GetDeb package updates

2009-12-10T13:11:00.000-08:00

Last week I have changed the package updates links organization, cleaned up the main updates fetching/filtering code and finally added the most frequently requested feature: RSS feeds.

With the new organization updates are available from the following links:
http://www.getdeb.net/updates/Ubuntu/$release/
Where $release = 9.04, 9.10.. or all to show updates for all releases.

The application specific pages are now http://www.getdeb.net/software/appname, right now they just provide the same info from /updates/, comments, ratings, etc will go into the /software/ section.

Let's fix bug #1 without regressions

2009-11-16T01:48:00.000-08:00

Linux usability on the Desktop OS has significantly increased, this brings two major problems:
1) Wider range of hardware and use cases, increasing the chances of problems with untested use scenarios,
2) Linux on the Desktop is moving from an hobby OS to a main workstation OS, regressions have a serious impact, they now affect some business or some people work

Ubuntu and Desktop distros in general are trying to resolve problem 1 by pushing new technology as faster as they can, but what about problem 2 ? Is it really being addressed ?

Some years ago the FAQ was "I have tried Ubuntu on my PC and there is no sound, no network and my device Y is not recognized", most of the times the root cause and answer was simple: "The current kernel drivers do not support your hardware yet" or "You need additional configuration to enable the driver" - additional work needs to be done, someone needs to improved/develop drivers and/or configuration tools.
These days, the FAQ is "My sound, network and graphics where working great until I did an upgrade", the answer "It's a regression" is not that easy to explain - work was done on some driver/configuration in order to improve it and it's failing on your case, possible reasons are: unknown problem(not tested), known (waiting for a fix), known (won't fix).

I hope we are not running in competition for hardware/features/whatever support with the leading OS.
The fix for bug #1 should not introduce such a severe regression as lack of reliability.

Forget the Koala fight and let's go Lynx

2009-11-04T13:44:00.000-08:00

There is a lof of debate whether Karmic Koala was a good or bad release, that is all a matter of expectations. In my opinion, we, the Ubuntu community did a great job but the outcome was not as good as some people expected.

I have read the following reply from Martin Pitt on the ubuntu-devel-discuss which makes me believe this was an accounted risked, part of the preparation for a much important goal which is 10.04 LTS:

" Well, the bug tracker is full of regression and other reports. Karmic
indeed was meant from the start as a "crack dump" release, with lots
and lots of new technology going into it. So in a way, it was the
Fedora of Ubuntu releases so far.

Perhaps we should have announced that more clearly...

At least we now have a full cycle ahead of us to do bug fixing. :) "

Unlike the usual blame game this was a much simpler and probably more realistic answer.

Instead of wasting energy debating if it really failed let's just try to fix it. Sometimes is not a technically failure and it is just lack of understanding on how things work, if that is the case we still need to fix it, it's a matter of improving awareness and communication.

GetDeb v2 Beta Available

2009-10-18T02:06:00.001-07:00

GetDeb v2 Beta is a major milestone on the GetDeb project development, it addresses most of the technical limitations of the current system/engine that we have been using for the last 3 years.
The major improvements are:

* Software installation using a GPG signed APT repository with mirror distribution
* Single click installation even for multi-package installs
* Automatic software updates (from the repository)
* New design - for the first time not based on a Ubuntu offical site
* Web based applications publishing

These changes will increase our ability to provide more and better packages in a safely manner.
Most of the effort was invested at the APT-Portal project, which is shared with Playdeb and because is open source it can be used by by other teams to deploy their own repository based portals.

Both the design and code base -rewritten in python- still need a lot of work, if you have any skills with Python/HTML/CSS/AJAX please join the developers team.

http://beta.getdeb.net
http://wiki.getdeb.net/apt-portal

Importing APT repository info into a SQL DB

2009-10-06T15:37:00.001-07:00

Hello,
in case you love repositories and databases there is a simple utility that allows you to export a remote APT repository metatada into a SQL database.
It was only tested with sqlite3 and MySQL, should work with any sqlalchemy supported DB .

First you need to get debfactory using bzr:

bzr checkout lp:debfactory

Now let's import the package info for Ubuntu Karmic (i386).

cd debfactory/bin
./apt2sql.py -d sqlite:///apt2sql.db \
http://archive.ubuntu.com/ubuntu/ karmic "*" i386

Check the packages count from the db:

sqlite3 apt2sql.db "SELECT COUNT(*) FROM package"