IBM Cell SDK/SDK for Multicore Acceleration on Debian/Ubuntu HOWTO
Saturday 15th December, 2007 @ 4:26 am
Today I battled with trying to get IBM’s Cell SDK 3.0 (now known as the SDK for Multicore Acceleration) installed on my Debian AMD64 machine. This would cause slight grief even with a regular Debian machine as IBM only formally supports Fedora and RHEL so only provides RPMs. This is worsened by the fact that a bunch of the packages aren’t even available from IBM but require perusal of some provided yum configuration files to find.
Firstly I recommend you download the CellSDK-Devel-Fedora_3.0.0.1.0.iso and CellSDK-Extras-Fedora_3.0.0.1.0.iso (current at time of writing) from here.
Then when you have downloaded them mount them somewhere on your filesystem.
mount -o loop $HOME/CellSDK-Devel-Fedora_3.0.0.1.0.iso /tmp/CellSDK-Devel-Fedora/
mount -o loop $HOME/CellSDK-Extras-Fedora_3.0.0.1.0.iso /tmp/CellSDK-Extras-Fedora/
We’ve downloaded the two RPMs by IBM. If you install these on Fedora they will provide you with some yum repositories for the Barcelona Supercomputing Centre. These will need to be downloaded manually as we aren’t on a RPM-based system.
If you are running Ubuntu Gutsy on x86_64 or x86 run:
apt-get install ppu-gcc ppu-gdb spu-g++ spu-gdb cell-programming-primer
If you are running Ubuntu Gutsy on the Cell run:
apt-get install cell-sdk
Otherwise run the following:
apt-get install wget #Install wget if it is not already
mkdir openrpm
cd openrpm
wget -l 1 -c -np -nd -r http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk3.0/CellSDK-Open-Fedora/x86_64/ -A .rpm
The aboveassumes you are on an x86_64 machine. If you are using the Cell, a regular x86 or a PPC 64-bit machine change the ‘x86_64‘ to ‘cbea’, ‘x86‘ or ‘ppc64‘ accordingly.
The following instructions apply regardless of your Debian/Ubuntu variant or architecture.
Next, if we are on x86_64, we want to create a nice little script to handily convert the i386 architecture files to amd64 so they will install nicely when we convert them to debs. This is safe as all the stuff these packages install gets chucked into /opt/.
I recommend we name it “fixcelldebsarch.sh” and stick it in your $HOME. This is only necessary if you are on x86_64 not x86 and (probably) won’t work on the Cell or another PPC64.
#!/bin/bash
#~/fixcelldebsarch.sh
OWD=`pwd`
for i in `ls -d */`
do
cd $i
sed -ie ’s/Architecture: i386/Architecture: amd64/’ debian/control
dpkg-buildpackage
cd “$OWD”
rm -r */ *.gz *.changes *.dsc
done
(Thanks to Jon for the fix to avoid mess when using symbolic links.)
Next we want to convert the various RPMs into DEBs for our Debian system so the dependencies are nicely handled and they can be uninstalled. Install the ‘alien‘ package if you have not already.
On the following lines replace the occurences of “/x86_64/” with ‘/cbea/’, ‘/x86/’ or ‘/ppc64/’ using the same criteria as above.
cd open
for i in ../openrpm/*.rpm; do alien --scripts $i; done
~/fixcelldebsarch.sh #Only needed on x86_64
dpkg -i *.debcd devel
for i in /tmp/CellSDK-Devel-Fedora/x86_64/*.rpm; do alien --scripts $i; done
~/fixcelldebsarch.sh #Only needed on x86_64
dpkg -i *.debcd extras
for i in /tmp/CellSDK-Extras-Fedora/x86_64/*.rpm; do alien --scripts $i; done
~/fixcelldebsarch.sh #Only needed on x86_64
dpkg -i *.deb
It should have been fairly obvious what was going on above. This should have installed all the necessary packages for the Cell SDK without breaking your system and allowing Debian/Ubuntu packages to override these versions.
I hope this all works; if anyone spots any typos, has any suggestions or needs any help then give me a shout!
Updated: Added Gutsy information. Thanks Bart!
Posted in Software Development
16 Comments »
Hey Arthur,
Thanks for the how-to.
Just to let you know there is an easier way to install the Cell SDK version 3.0 on ubuntu gutsy since it’s in the repositories.
On a x86 or x86_64 system use:
“apt-get install ppu-gcc ppu-gdb spu-g++ spu-gdb cell-programming-primer”
On a Cell system with ubuntu gutsy you can use:
“apt-get install cell-sdk”
Also note that this only contains the open-source licenced parts of the Cell SDK. Some of the major libraries, like libsync and DaCS, are not distributed by the Barcelona Supercomputing Centre.
I’ll look into an automated way to get the IBM libraries and the licencing specifics, since I’m assembling a new Celluntu live DVD with SDK 3.0 preinstalled.
Greets
Bart
Comment by Bart Cerneels — Monday 17th December, 2007 @ 9:22 am
Cheers Bart (It’s Mike by the way, not Arthur!).
I’ll add in the Ubuntu method this evening. Does Ubuntu install the AMD64-native versions of all the open stuff?
Comment by Mike — Monday 17th December, 2007 @ 10:53 am
It does seem so, yes.
Comment by Bart Cerneels — Monday 17th December, 2007 @ 12:21 pm
Hi Bart,
That’s great news about Cell programming on the PS/3. I’m new to Cell programming, so I’m curious: does not having libsync and DaCS hurt you much when programming a PS/3’s Cell?
I’ve been trying to get IBM’s SDK 3.0 running on FC7 on my PS3. But the SDK 3.0 installer is pretty weak, which leaves me really wishing/hoping that I can get by with Ubuntu on the PS3 and just running “apt-get install cell-sdk”.
Cheers,
Christian
Comment by Christian Convey — Thursday 20th December, 2007 @ 9:21 pm
Nice guide, but I can’t find the openrpm folder mentioned in:
for i in ../openrpm/*.rpm; do alien –scripts $i; done
Where do I find that one?
Without the package I get this error:
Unpacking of ‘../Devel/cell-documentation-3.0-5.noarch.rpm’ failed at /usr/share/perl5/Alien/Package/Rpm.pm line 153.
find: invalid predicate `-’
I run the stock gutsy version.
Comment by KenKen — Saturday 5th January, 2008 @ 5:36 pm
Sorry KenKen, forgot to add the “mkdir” for openrpm and Wordpress replaced the double-dashes for alien with a single one. Should work now.
Comment by Mike — Saturday 5th January, 2008 @ 6:15 pm
Thanks, I presumed that my second failure was related to the first one, but it was not.
After installing the openrpm stuff, Ubuntu complained that I was downgrading all the packages, so the following lines should be removed from the script, if you use apt-get to install:
cd open
for i in ../openrpm/*.rpm; do alien –scripts $i; done
~/fixcelldebsarch.sh #Only needed on x86_64
dpkg -i *.deb
My next bump is the rpm’s on the CD are not recognized as RPM files:
~>alien –script alf-cross-devel-3.0.0-9.noarch.rpm
argument is not an RPM package
Unpacking of ‘alf-cross-devel-3.0.0-9.noarch.rpm’ failed at /usr/share/perl5/Alien/Package/Rpm.pm line 153.
find: invalid predicate `-’
Debugging a bit gives:
~>rpm2cpio alf-cross-devel-3.0.0-9.noarch.rpm
argument is not an RPM package
So, for some reason rpm2cpio does not recognize the RPM package format for all RPM’s on the Devel and Extra iso, but does recognize the openrpm stuff.
Do you have any suggestions?
Comment by KenKen — Monday 7th January, 2008 @ 10:48 am
The first point: You only want to remove it on Ubuntu if you are installing the “cell-sdk” package.
The second point:
No idea. My only guess is that you have either got too old a version of your rpm utilities or that they are corrupted.
Comment by Mike — Monday 7th January, 2008 @ 6:53 pm
Hi Mike,
came across your small article while looking for ways to install the Cell under Ubuntu, which works a treat btw. Just one thing, the first lines for converting the packages downloaded are only needed if you haven’t installed the cell-sdk under Gutsy or a Cell powered box, right?
It sort of confused me at first (am running Gutsy so I can cross compile to my PS3) and it may confuse some other people if they try the first conversion step and get some errors. Maybe it’d be a good idea to point out that the first package conversion is only needed if if you’re not running Gutsy or a Cell powered box.
cheers
James
Comment by James — Sunday 17th February, 2008 @ 4:57 pm
Hi Mike,
thank you for this nice howto. I ran into some problems, first I got some problems combining the packages from apt-get and the ones from the CD. This is solved by only using packages from the CD.
The next problem is a bit more serious. Something wrong happened when fixcelldebsarch.sh tried to change directory, making it alphabetically deleting directories in my home directory. Fortunately they did not include any important files:) I would therefore suggest to use a more safe version, e.g.
#!/bin/bash
#~/bin/fixcelldebsarch.sh
origDir=`pwd`
for i in `ls -d */`
do
cd $i
sed -ie “s/Architecture: i386/Architecture: amd64/” debian/control
fakeroot dpkg-buildpackage
cd “$origDir”
done
Comment by Jon — Wednesday 12th March, 2008 @ 11:13 am
Thanks Jon, updated.
Comment by Mike Arthur — Wednesday 19th March, 2008 @ 1:35 am
Mike!
Converting the .rpm from openrpm into .deb format succeeds for the most of them, but some fail by reporting:
Package build failed. Here’s the log:
dh_testdir
dh_testdir
dh_testroot
dh_clean -k -d
dh_installdirs
dh_installdocs
dh_installchangelogs
parsechangelog/debian: warning: debian/changelog(l7): badly formatted heading line
LINE: - Fix PR 38589 Generate inline code for int to double conversion by default,
parsechangelog/debian: warning: debian/changelog(l8): found change data where expected next heading or eof
LINE: and call libgcc function with -Os.
Can’t locate object method “init” via package “Dpkg::Changelog::Entry” at /usr/share/perl5/Dpkg/Changelog/Debian.pm line 258, line 8.
dpkg-parsechangelog: failure: changelog parser /usr/lib/dpkg/parsechangelog/debian gave error exit status 9
dh_installchangelogs: changelog parse failure
make: *** [binary-arch] Error 1
find: spu-gcc-4.1.1: No such file or directory
Ups! Any ideas?
x86, debian/testing, sdk3.0
Frank
Comment by Frank — Saturday 5th April, 2008 @ 11:14 am
I forgot to mention, that these .rpm are affected:
spu-gcc-4.1.1-107.i686.rpm
spu-gcc-c++-4.1.1-107.i686.rpm
spu-gcc-debuginfo-4.1.1-107.i686.rpm
spu-gcc-fortran-4.1.1-107.i686.rpm
All others .rpm in ./openrpm are processed by alien without any complains, besides:
warning: ../openrpm/libspe2-cross-devel-2.2.0-91.noarch.rpm: Header V3 DSA signature: NOKEY, key ID 77550217
Maybe it does not work on Debian/Testing and it must be an Debian/Ubuntu?!
Best!
Frank
Comment by Frank — Saturday 5th April, 2008 @ 4:36 pm
Thanks for the howto, Mike.
I am having some trouble installing SDK 3.0 on Ubuntu Fiesty Fawn. I followed your instructions, and encountered a few errors.
Firstly, when running alien, I get a bunch of warnings that look like this:
warning: /media/cell/x86/trace-cross-devel-3.0-38.noarch.rpm: Header V3 DSA signature: NOKEY, key ID 9ac02885
Then, when running dpkg on the open directory, I failed to install one of the packages:
Unpacking numactl (from numactl_0.9.10-2_i386.deb) …
dpkg: error processing numactl_0.9.10-2_i386.deb (–install):
trying to overwrite `/usr/lib/libnuma.so.1′, which is also in package libnuma1
Errors were encountered while processing:
numactl_0.9.10-2_i386.deb
Also, I get this during dpkg in the devel directory:
Setting up cellide (3.0.0-19) …
chmod: cannot access `/opt/ibm/systemsim-cell/run/cell/linux’: No such file or directory
Anyway, I charged through. A quick glance at /opt/ibm/systemsim-cell/ and there is nothing there except an empty directory called images.
Any suggestions would be appreciated.
Comment by Scott — Wednesday 9th April, 2008 @ 2:24 am
According to a bug in /usr/share/perl5/Dpkg/Changelog/Debian.pm
the packages:
spu-gcc-4.1.1-107.i686.rpm
spu-gcc-c++-4.1.1-107.i686.rpm
spu-gcc-debuginfo-4.1.1-107.i686.rpm
spu-gcc-fortran-4.1.1-107.i686.rpm
failed for me too on an Ubuntu 8.04 x86.
The patch from here seems to fix this problem:
https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/206790
Comment by daniel — Monday 14th April, 2008 @ 11:28 pm
Hello
I am new to Linux & having trouble with conversions of rpms to debs for CellSDK2.0.
OK can you explain in which directory is the following script supposed to be saved.
cd open
for i in ../openrpm/*.rpm; do alien –scripts $i; done
dpkg -i *.deb
cd devel
for i in /tmp/CellSDK-Devel-Fedora/x86_64/*.rpm; do alien –scripts $i; done
dpkg -i *.deb
cd extras
for i in /tmp/CellSDK-Extras-Fedora/x86_64/*.rpm; do alien –scripts $i; done
dpkg -i *.deb
I have Core 2 Duo & plan to install i386 Ubuntu.
And your tutorial talks about mounting the iso image first. Where is that being used?
In the iso image that i have downloaded, there’s no folder called openrpms, though there is an rpms folder & separate folders for each of the architectures (x86, x86-64,…)
I am not able to follow hoe to proceed.
Comment by Rohit Banga — Wednesday 4th June, 2008 @ 6:54 am
Leave a comment