Other Help Topics :: HOWTO for creating .uci extensions?

I'm confused. I haven't found any other reference on how to create a .dsl package other than from debs.

Shouldn't there be consistency on how to create an extension?

So, how do you create a .dsl from source and how would you create an .uci from debian packages?

A Linux distro is all about package management, otherwise people would be happy with a shell and a compiler, like in 1992. Projecting this in a 5 year time seems too confusing to me.

There are three type of extensions: *.dsl, *.tar.gz, and *.uci (there was a *.ci but they aren't really around anymore). The difficult part about creating extensions is finding the files that go into them. That's why cbagger's deb2dsl script is so popular: it finds the files you need on it's own. That said, here are the mechanics of creating extensions, followed by some tip and tricks on making menus and icons, and finding files.

The easiest extension to create is the *.dsl extension. Its files can go in any directory. The disadvantages are they load the files into memory (bad for low memory systems) and they can overwrite files that already exist (someone could overwrite an important file like /etc/mtab). Once you have a list of files (mine is called files) you create a *.dsl package using the following command line:

tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files

The second type of package is a *.tar.gz. It is created almost identically to a *.dsl file except the files it contains must all be located in the /opt, /home, /etc, /var, and /tmp directories. Again with a list of files called "files" you create the package using this command line:

tar -czvf mypackage.tar.gz --numeric-owner --no-recursion -T files

The third type of package is a *.uci. It also has the restriction that files must be in the /opt/mypackage (replace mypackage with a name you choose), /home, /etc, /var, and /tmp directories, but it also requires that the program can run correctly if /opt/mypackage is read-only. Many programs don't write their executable files so you could be OK, but you need to check it out. It looks harder to make, but it's fairly easy if you build a working *.tar.gz extension version first. I don't know about you, but I never get these right on the first shot. Something always needs tweaking and *.dsl and *.tar.gz extensions are far easier to tweak. If you do need to write to the /opt directory you'll still have a working tar.gz extension. The extension name, the directory under the opt directory, the menu file name, and the icon name must all be same. In my case this is "mypackage."

The same way dsl and tar.gz files are a gzipped tar bundle, a uci file is a compressed file system created using mkisofs and create_compressed_fs. It is similar to the /KNOPPIX/KNOPPIX file which is created in a similar way. The compressed system is mounted to a directory create in /opt, and because the compressed file system is read only, you can't write to that directory. The uci's root is your tar.gz package's /opt/mypackage/ directory. Inside that compressed file system is anything you had originally in that directory, plus a user.tar.gz file which contains everything that needs to be in the /home or /tmp directories (it's your original mypackage.tar.gz without the opt/ directory). It sounds a lot more complicated than it really is.

Lets say you have an extension named mypackage.tar.gz and it works the way you want it to. The following lines will convert your tar.gz extension into a uci extension:

mkdir work
sudo tar -xzvf mypackage.tar.gz --same-ower -C work
tar -tzf mypackage.tar.gz | grep -v "^opt" | tee work/user.files
cd work
sudo tar -czvf opt/mypackage/user.tar.gz --numeric-owner --no-recursion -T user.files
cd opt
mkisofs -R -hide-rr-moved -cache-inodes -pad mypackage/ | \
create_compressed_fs - 65536 > ../../mypackage.uci

Here is a first cut at a script that converts a *.tar.gz package into a *.uci package. It does absolutely no checking to see that you have kept the same names between the package, the /opt/package, the tmp/mydsl.menu/package, and the home/dsl/.xtdesktop/package.lnk files.

Code Sample

#!/bin/bash
#targz2uci.sh

if [ -z $1 ]
then
echo no file name to convert!
echo "usage: targz2uci.sh mypackage.tar.gz"
else
PACKAGENAME=`echo $1 | sed 's/\.tar\.gz$//'`
mkdir work
sudo tar -xzvf $1 --same-owner -C work
tar -tzf $1 | grep -v "^opt" | tee work/user.files
cd work
sudo tar -czvf opt/$PACKAGENAME/user.tar.gz --numeric-owner --no-recursion -T user.files
cd opt
mkisofs -R -hide-rr-moved -cache-inodes -pad $PACKAGENAME/ | \
create_compressed_fs - 65536 > ../../$PACKAGENAME.uci
fi

I've been told that all files in /opt need to be owned by root and the group needs to be root as well. I can neither confirm nor deny this. I'm just passing it on, but it would be nice to know which way it is and then I'll go back and edit this...

----------------------------------------------------

Creating menu entries isn't hard, but I've found it's easier to do after you've already made a *.dsl or *.tar.gz extension. What you need to do is create a file in a tmp/mydsl.menu directory that has the same name as you package has, capitalization and spelling count. So if my package was called mypackage.dsl, my menu would need to be called mypackage. The menu file is just a text file so create and edit it using whatever you want: beaver, vi, nano, whatever. The entries are fairly straightforward. For a single menu entry the form is:

[exec] (The string the menu displays) { command the menu runs }

to create submenus the form is:

[submenu] (Submenu name)
[exec] (item1 string) { command for item 1 }
[exec] (item1 string) { command for item 1 }
[end]

Or go crazy and have submenus in submenus if you need to:

[submenu] (Submain0 menu name)
[submenu] (Submain1 menu name)
[submenu] (Submain2 menu name)
[exec] (item1 string) { command for item 1 }
[end]
[end]
[end]

a common way to run cli programs is in an xterminal. To bring one up and run your application in it the form is:

[exec](my menu name){aterm +tr -T "Window Title" -e /opt/mypackage/bin/myprogram}

So to add a menu to your new package, you world create 2 directories, create the menu item, run chmod 644 on the new menu item (it will not work if you leave out that chmod step), and add it to your extension. The only hard part is that you can't add files to compressed tars, so you'll need to use gzip to uncompress and recompress the tar. To do this the file name must end in tar.gz, so you'll have to rename your dsl package (not needed for tar.gz packages). Like so:

mkdir tmp
mkdir tmp/mydsl.menu
beaver tmp/mydsl.menu/mypackage
# add your menu entries to your menu file
chmod 644 tmp/mydsl.menu/mypackage
mv mypackage.dsl mypackage.tar.gz # if it's a dsl package
gunzip mypackage.tar.gz
tar -rzvf mypackage.tar --numeric-owner --no-recursion tmp/mydsl.menu/mypackage
gzip mypackage.tar

Some people, myself included, find it easier to unpack their extension to a directory, create the tmp/mydsl.menu/mypackage menu item there, add that file to the files list, and then repack. It takes more room but it's easier:

mkdir work
cp mypackage.dsl work
cd work
sudo tar -xzvf mypackage.dsl --same-owner
mkdir tmp
mkdir tmp/mydsl.menu
beaver tmp/mydsl.menu/mypackage
# add the menu items, save and close editor
chmod 644 tmp/mydsl.menu/mypackage
tar -tzf mypackage.dsl > files
# add tmp/mydsl.menu/mypackage to the files list using beaver or whatever
tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files

----------------------------------------------------

Making icons requires you to create a mypackage.lnk file and an image file for the icon in a home/dsl/.xtdesktop directory. The icon and image name must be the same as the package name.

the format for the lnk file is:

Code Sample

table Icon
Type: Program
Caption: My Program
Command: /opt/mypackage/bin/myprogram
Icon: .xtdesktop/mypackage.png
MenuCommand1: my menu text:/opt/bin/program2
X: 400
Y: 400
Status: anchor
end

MenuCommand1: is kind of neat. You add items here and if the user right clicks on your icon, a menu pops up and they can choose to run this command. Separate your text from the command with a ':' See the /home/dsl/.xtdesktop.Beaver.lnk for an example. Right click on the Beaver icon and you get the option of starting beaver as root.

The icons are normally 48x48 pixels but they can be bigger or smaller, and can be one of the following types: gif, png, jpg, ppm, or xpm. You can create the icon in XPaint if you want to, but if you want an alpha channel (the see through parts of the icon) you'll need a stronger program.

Following a similar procedure to the above menu steps, you could do:

mkdir work
cp mypackage.dsl work
cd work
sudo tar -xzvf mypackage.dsl --same-owner
mkdir home
mkdir home/dsl
mkdir home/dsl/.xtdesktop
beaver home/dsl/.xtdesktop/mypackage.lnk
# edit with beaver or whatever to change names and positions
# create or copy an icon into home/dsl.xtdestop
# change read-write-execute permissions to -rw-r--r--
chmod 644 home/dsl/.xtdesktop/mypackage.*
tar -tzf mypackage.dsl > files
# add /home/dsl/.xtdesktop/mypackage.lnk and mypackage.png to the files list
tar -czvf mypackage --numeric-owner --no-recursion -T files

----------------------------------------------------

Finding the files for the file list, as I said in the beginning, is the hardest part. A lot of the time I'll use the find command to make it easy on myself. I'll load the gnu-utils.dsl package first so I can use the better find command than the one that comes with the busybox stuff in dsl. Since the normal order of operations in compiling is:

./configure
make
sudo make install

I'll create a temp file just before the make install step and the search for files created after the install step, which tells me what was installed.

./configure
make
touch mymarker
sudo make install
sudo find / -not -type 'd' -cnewer mymarker | grep -v "\/proc\/" | tee files

Remove the leading /ramdisk from each line of the file (use beaver's replace or whatever). The first time you create the archive, you'll need the leading / to tell tar to get the files from the root directory. Then use the
tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files
command to create a dsl package like you did above then untar the package to a work directory using
tar -xzvf mypackage.dsl --same-owner -C work
You can then use the tar -tzf mypackage.dsl option to get the list of files back. You don't want a forward slash "/" at the start of the lines in the files list once you start tarring and untarring from the work directory.

You also want to be careful that you dont include any directories that already exist. While it doesn't delete the existing on, it can change the ownership/permissions and that can foul things up. So look at the files and check for things like bin/ or etc/ all by themselves with nothing after them.

You need to be careful to preserve the owners and groups of the files in your archives. If you ever unpack and then repack your extension, say when you're adding menus and icons, make sure to use the --same-owner switch in the tar command to preserve owners and groups. I have read elsewhere in a useful post by ke4nt1 that the owner.group for files in your extension directories should be:

/etc = root.root
/usr = root.root
/home/* = dsl.staff
/tmp/* = dsl.staff
/opt = root.root (usually)

Another great piece of advice from that post was to look at the files in your extension using tar -tzf mypackage.dsl > files and comparing them to files already included in dsl. Are you overwriting any existing files? Do you really need to?

----------------------------------------------------

If programs won't run because they're missing libraries, you can use the ldd command to list what libraries the program wants to load and then add those libraries to your package. You need to be careful because sometime ldd gives you a library that is really a link to another library. You end up adding the library but you only added a link that points to something you didn't put into the package. You need both the links and the real libraries in your package. deb2dsl is a great script, but it includes every library that the apt-get said you needed. Sometimes you only need parts of what you actually downloaded. ldd can tell you that if you look at all of the executables in your package.

----------------------------------------------------

I would never put the following files in extensions. They are always getting caught in the find command but I know they will overwrite other existing files and will be bad.

/etc/mtab
/etc/fstab
/etc/ld.so.cache
(you can save /etc/ld.so.conf but make sure your app runs sudo ldconfig)
/lib/modules/2.4.26/modules.dep
/lib/modules/2.4.26/modules.generic_string
/lib/modules/2.4.26/modules.ieee1394map
/lib/modules/2.4.26/modules.isapnpmap
/lib/modules/2.4.26/modules.parportmap
/lib/modules/2.4.26/modules.pcimap
/lib/modules/2.4.26/modules.pnpbiosmap
/lib/modules/2.4.26/modules.usbmap
(your program can run depmod and these will be set correctly if it's needed)

----------------------------------------------------

The last piece of advice I can offer is in the creation of tar.gz and uci packages. It is sometimes possible to compile your program like this:

./configure --prefix=/opt/mypackage
make
make install

This makes all of the executable and library files created by the program go into the /opt/mypackage directory and the executables will know to look there for what they need.

----------------------------------------------------

EDIT:

hit return and posted before I was done.
Added that /etc can be in tar.gz and uci extensions per Mikshaw's post.
Added the part about removing /ramdisk from the start of lines in the file list
Added info about icon sizes
Added chmod 644 for icons, owner.group info, and warning about overwriting existing files

Yeh, like that wasn't enough. Here is a revised script to go from a *.tar.gz to a *.uci file, hopefully with some useful error messages:

Code Sample

#!/bin/bash
#targz2uci.sh

if [ -z $1 ] || ( echo $1 | grep -qv "tar.gz$" )
then
echo "usage: targz2uci.sh mypackage.tar.gz"
exit -1
else
PACKAGENAME=`echo $1 | sed 's/\.tar\.gz$//'`
mkdir work
sudo tar -xzvf $1 --same-owner -C work
tar -tzf $1 | grep -v "^opt" > work/user.files

# Check extension basename against the name in the opt directory
if ! [ -d work/opt/$PACKAGENAME ]
then
echo "uci files must have their base name and the name"
echo "of the directory they create in /opt the same"
exit -1
else
echo "base name check OK"
fi

# Check that their are only tmp, var, etc, and home directories besides the opt
if ( grep -v "^home" work/user.files | grep -qv "^etc" | grep -qv "^tmp" | grep -qv "^var" )
then
echo "Directories other than home, etc, tmp, and var found in archive."
echo "only opt, home, etc, tmp, and var directories are allowed in uci files!"
exit -1
else
echo "check for home, etc, tmp, \& var OK"
fi

# Check the menu file's name against the basename
if [ -d work/tmp/mydsl.menu ] && ( ls work/tmp/mydsl.menu/ | grep -qv "^$PACKAGENAME$" )
then
echo "menu name not the same as the extension"
exit -1
else
echo "menu file name OK"
fi

# Check the icon file names against the basename
if [ -d work/home/dsl/.xtdesktop ] && ( ls work/home/dsl/.xtdesktop/ | grep -qv "^$PACKAGENAME\." )
then
echo "icon items do not match extension name"
exit -1
else
echo "icon file name OK"
fi

cd work
sudo tar -czvf opt/$PACKAGENAME/user.tar.gz --numeric-owner --no-recursion -T user.files
cd opt
mkisofs -R -hide-rr-moved -cache-inodes -pad $PACKAGENAME/ | \
create_compressed_fs - 65536 > ../../$PACKAGENAME.uci
fi

EDIT:

added quotes around echo strings per Mikshaw's suggestion
added the ability to use the /etc directory, again thanks Mikshaw

a couple of minor things about the script....

1) Breaking up an echo command for a multi-line echo into more than one command makes reading the script a little confusing (only a little). I suggest one of these options...
Use quotes for each echo line, to make what is happening more apparent, visually.

Code Sample

echo "uci files must have their base name and the name"
echo "of the directory they create in /opt the same"

Or use a single echo that spans multiple lines. This one requires quotes, but should pass the linebreaks without trouble

Code Sample

echo "uci files must have their base name and the name
of the directory they create in /opt the same"

2) What about /etc? A *.tar.gz extension can install files to /etc without the need to run mkwriteable, even though space is apparently limited.

Thanks Mikshaw. I had no idea tar.gz and uci files could contain files in the /etc directory, but I just tried it out and you're right. I thought it might have been because *.tar.gz extensions were supposed to be safer that the regular dsl ones, but apparently there isn't a check for that. I'll modify my posts above to reflect the ability to include those directories in tar.gz and uci extensions. Any other advice on the post or the script is welcome. I did my best but I'm sure there are omissions and errors.

Next Page...
original here.