Donations...

A couple of people asked for a donate button of some sort. I hesitate because of the sudden complexity of things when money is involved. Here is what I think would be best:

• Donate your time and skills if you can rather than money.
• If you cannot do so, please consider sponsoring something we could directly use: Such as a USB serial cable (about $45 for parts from Sparkfun), or perhaps upgrading the Slicehost slice we're borrowing from pumpkin ($60 per month, though it would only be useful if we could get enough money for at least six months), used or broken devices for testing, etc. Or something like a pizza for one of our contributors on a late night hacking binge (we will distribute food donations on a round-robin basis =P)
• My paypal address is my gmail e-mail address (planetbeing). If you do make a donation, please specify exactly what it is for. A gigantic slush fund is something that I wish to avoid.
  

For the record, contrary to what was published by some media sources, the iphonelinux group is not associated with the group commonly known as the "iPhone Dev Team". I'm a member of the Dev Team, and other members sometimes lends assistance or advice, but this is an entirely separate project. The Dev Team does not accept any donations whereas this project tentatively will.

Updating and uninstalling openiboot

It's actually slightly problematic to use the "Update Firmware" feature of iTunes with openiboot installed, so I updated openiboot with an uninstall facility.

I uploaded the binaries and instructions necessary to update openiboot (and then uninstall it if you wish) to: http://www.iphone-dev.org/planetbeing/openiboot-uninstall.tar.gz

Linux on the iPhone!

I'm pleased to announce that the Linux 2.6 kernel has been ported to Apple's iPhone platform, with support for the first and second generation iPhones as well as the first generation iPod touch. This is a rough first draft of the port, and many drivers are still missing, but it's enough that a real alternative operating system is running on the iPhone.



What we have:

- Framebuffer driver
- Serial driver
- Serial over USB driver
- Interrupts, MMU, clock, etc.

What we have in openiboot (but hasn't been ported yet):

- Read-only support for the NAND

What we don't have (yet!):

- Write support for the NAND
- Wireless networking
- Touchscreen
- Sound
- Accelerometer
- Baseband support

The current userland we're using, in the interest of expedience, is a Busybox installation created with buildroot, but glibc works fine as well, and we're going to build a more permanent userland solution.

A demonstration video can be seen here: http://www.vimeo.com/2373142

Instructions here: http://www.iphone-dev.org/planetbeing/LINUX-README.txt

Download here: http://91.186.26.18/iphone/files/iphonelinux-demo.tar.gz (look for mirrors in the comments)

EDIT: The instructions are missing the step that you have to select openiboot console from the menu before performing the "sudo ./oibc" step. Just be aware you have to do that if it seems like you're not getting a response from the oibc client.

Project lead: planetbeing

Contributors: CPICH, cmw, poorlad, ius, saurik

If you're experienced with hacking/porting Linux and especially if you're experienced with porting Android, I'd definitely like to hear from you. Come chill in the #iphonelinux channel on irc.osx86.hu. Thanks. :)

EDIT: I was asked a couple times by people who wanted to donate (financially) to the project. I made a post discussing this possibility, if you are interested.

Poorlad's menu implemented; Porting issues resolved

Yesterday night, I merged in a branch I was working on for poorlad's menu. A version of that beautiful menu is now in Git. His menu included a version string at the bottom. We didn't have any way to keep track of versions and builds before, so this was actually a good idea that I had to implement. Because I didn't want to implement support for non-fixed width fonts, or add another space-consuming font, I just used the console font I was already using for that part. I also had to brighten the gradient on the bottom of the screen, since it was basically invisible due to gamma issues otherwise.

The border between the gradient and the "black" is clearly visible on my device. This is probably because of a gamma issue. When poorlad comes back, we can ask him to calibrate it more.

Otherwise, it looks pretty good! In order to make this possible, I added in stb_images.c, a great tiny little image library that can read PNG, JPEGs and even PSD files and does zlib decompression as an added bonus. This will be a great help if we decide to change things or need to add more stuff that consumes a lot of space. I also added in a basic function to perform alpha blending (albeit comparatively slowly).

Sadly, while I was busy making these changes, ius from IRC actually begun to implement poorlad's menu without me knowing about it, so we ened up duplicating each other's efforts. He was able to compile in zlib and libpng, but the cost was to inflate the final binary to 347 KB. Whereas taking out the old menu images, and adding small, compressed PNGs and the stb_images library instead actually made openiboot smaller than it was before! His decision to preblend the images, rather than attempt alpha blending on the device, was probably more optimal from a performance perspective.

Steven Troughton-Smith told me on Twitter that he has actually implemented his own boot menu as well. I'm not sure if he used the new PNG code or not, but the new code makes it pretty easy for a competent programmer to add in whatever menu they would like. I'd tell everyone to skin away, but we should keep as few wild branches of this project as possible, since everyone randomly installing openiboot just for kicks (especially a modified version) and then coming to us (read: me, ultimately) for support is something we don't have the resources to handle at this moment.

On the porting side, the issues with installation, optimizing NOR access on iPhone 3G, NAND access on a few devices all seem to have been fixed, so we can basically scratch the first two items off of the list I put up in the last post. I'm pleasantly surprised at how relatively easy it was.

Anyway, now for the kernel. Well, if I don't get distracted by writing to NAND.

Porting to iPhone 3G and iPod touch

Hey guys,

The lack of updates for the past few days is because many of you decided to visit us in IRC, thus enabling work to be done on porting openiboot to the iPod touch and the iPhone 3G (in particular because I don't have an iPod touch at the moment).

I'm pleased to report that everything now seems to be working on the iPhone 2G and the iPhone 3G (albeit NOR read/write on the iPhone 3G is unoptimized and is unacceptably slow). There is apparently an outstanding issue with the NAND ECC on some (?) iPod touchs, and also some people can't seem to actually install openiboot to NOR on both iPhone 2G and iPod touch. Unfortunately, the problem is that these things happen on devices that I don't have physical access to, and IRC is often a frustrating medium for communicating with testers. I'm confident these issues will be resolved soon, though.

So, current simultaneous projects:

1. Resolve openiboot porting issues
2. Implement poorlad's boot menu
3. Work on write support for FTL

After at least one of those things are done, we'll be working on the Linux kernel.

NAND filesystem now readable!

Amazingly enough, the FTL_Read stuff from last night was pretty much correct! After that, it was relatively trivial to port over the HFS+ code I've already written (which was in pure C... finally that [fail] design decision has been vindicated =P).

As you can see in the screenshot below, with the latest Git revision, you can browse the filesystem from openiboot!


Next on the list is to port openiboot over to the iPod touch and iPhone 3G. It's probably just a matter of putting in different numbers for the GPIO ports, but we'll see.

After that, I will implement poorlad's bootmenu (which everyone seems to like).

After that, well... We have pretty much all the devices now, so we'll start looking at the Linux kernel. If you're a Linux kernel guy who would be willing to help (preferrably you have experience porting Linux to new ARM platforms), please leave a comment here. I can do most of the muscle work, but it'd be nice if someone can show me how to set up the source tree properly for the new port.

FTL

I don't know how I was talked into reversing a FTL, but we're actually on our way. I've managed to enlist the aid of CPICH (who has been helping with the lower layers as well, he's our human HexRays) and just recently, pumpkin, who you will know from the Dev Team. pumpkin will be the heavy support that's necessary to take down _FTLRestore, which is the most complex function I've seen in 1.1.4 iBoot (and I've pretty much have seen all of it). pumpkin is very good, so this task should be now be cut down to "fairly difficult" from "completely impossible".

The strategy so far has been me methodically hacking through the functions in the order that they are called, completely decompiling them, understanding them, and assimilating them into openiboot. Toward this end, I've been working on FTL_Open, which is a fairly large (but as it turns out, boring) function, but has been useful in enlightening us on several of the large data structures FTL uses.

Meanwhile, CPICH works on functions ahead of me, so that when I reach them, a lot of the thorny underbrush has been cleared out and my job becomes much easier and faster. Toward this end, he has been working on FTL_Read, which uses the data structures that the now-completed FTL_Open should populate.

_FTLRestore is sort of a "bonus", since it's not normally called if the iPhone was shut down normally and everything is cleaned up. However, since recovering faulty data structures require all redundancies to be exploited, reversing this would let us gain a lot of insight into how the FTL works. It's also, naturally, an enormously complex function, and hence I wisely delegated it to pumpkin. =P (We will probably end up working on it together)

The one thing that troubled me was that the code we were reversing is for 1.1.4 whereas we primarily need it to work on 2.0. However, due to the fact that I had it better mapped out than the 2.0 iBoot, and the fact that the equivalent 2.0 code was much more complex (lots of function pointers flying around, and a weird switch idiom I haven't quite figured out yet), We decided to stick to the 1.1.4 iBoot.

After completing FTL_Open, I had a bit of a panic when I discovered it did not work at all on my 2.1 phone, and I could not find any obvious bugs with it. This might've meant that all our work on 1.1.4's FTL was for naught. Forgoing sleep, I tore through the 2.1 iBoot, locating the analogues to my already reversed 1.1.4 functions (I had given up trying to trace through the function pointers the first time around), and called them directly with my special version of iBoot (patched so that one of the commands was able to call arbitrary iBoot functions with arbitrary arguments). I managed to find a couple of bugs with my VFL code, and after having fixed them, FTL_Open appears to have worked. I think. It just finds and reads several data structures from NAND. It remains to be seen if I'm even reading the right thing.

Now for some sleep.

Why iPhone Linux?

This is a post I wrote a long time ago, when this blog was first conceived. I decided to hold off on posting it, because I thought it'd be better to do some technical posts before waxing philosophically. I think it is still appropriate, so as we work on reverse engineering the NAND FTL, here's some food for thought.

Porting Linux to the iPhone is an arduous project. We will be trying to develop an entire suite of device drivers for undocumented hardware and then attempt to run a full-fledged operating system on it. This thread speculates "10 days" or "3 hours" as the amount of time it'd take to get Linux up and running on the iPhone. Perhaps this figure would be accurate on a x86 platform, or other platforms with hardware for which device drivers are already written or for which at least documentation is available, but we have no such luck on the iPhone.

This comment on a O'Reilly Radar article about NerveGas's iPhone Open Application Development book says, with perhaps a little too much vitriol for my taste, that developers should not waste time on the iPhone, a closed platform, and spend time more productively on OpenMoko or Android: truly open platforms. Apple should thus be punished for not making the iPhone open. His point is well-taken though. Reverse engineering Apple's code is inefficient and ought to be unnecessary. Why do I bother when I can just develop on an open platform instead with no such wasted effort?

Finally, I have faced skepticism even from my fellow Dev Team members when I first talked about this project. The iPhone already has a perfectly serviceable operating system that we can develop on. Why does it need another one? Sure, Linux might be cool, but what practical use would it have? How does it justify the tremendous amount of effort that would need to be put in?

So. Why do I bother? Why should we bother?

Part of the answer is that I don't choose which platform I hack on based on how hackable it is. I choose it based on how much I like it. I don't own an OpenMoko device; it simply doesn't look as polished as the iPhone, and support is lacking for it. It wouldn't make sense to buy it to use it, only to buy it to hack on it. While this may work for other people, it's simply not the way a (relatively) starving college student does things. As for the Android, I'm not too convinced about how amazing it will be from the videos I've seen and besides: It doesn't even exist yet! In general, the more people use a device, the more hackers use it, and thus the more it is hacked on. Usability frankly trumps hackability.

The other part of the answer is that iPhone Linux will actually be of tremendous value. There will be no more need to port applications over: The applications already run on the iPhone! Also, with a familiar kernel, we can do all kinds of things I've wanted to do: doing security related work with the wi-fi for example. Plus, knowledge that we are gaining/will have gained about the iPhone hardware will be of incredible practical value to the homebrew iPhone community. We've always wanted to be able to plug in the iPhone as a simple USB mass storage device. With USB and NAND FTL drivers, we can actually implement this ourselves.

Perhaps my most important point is how iPhone Linux will affect the various open platforms in development. The iPhone has revolutionized the way the market thinks about mobile computing and now several mobile platforms are in development: OpenMoko, Google's Android, and Mobile Ubuntu (thought the last is not targeted for phones). All of these projects are based on Linux, and "based on Linux" means that, by definition, they "use the Linux kernel" and the Linux kernel is exactly what we're porting. As long as the kernel works, the rest of the operating system will barely need to be touched at all! (fine print: provided that the working configuration of the kernel can support all the features the userland requires).

Imagine OpenMoko on the iPhone. Android on the iPhone. Ubuntu Mobile on the iPhone. Consumers will have choice, and not some Linux-hippie idealistic choice-for-the-sake-of-choice choice: All of these platforms have major momentum behind them and it is very possible they will end up being better than the iPhone's platform (have better UI, more application support, etc.). Also, imagine what it will mean for the developers of these platforms: A ready userbase of millions of users. If many people can already install and try out one of these platforms, it'll be far easier to attract users to buy the hardware, and developers to develop for the platform. Thus, I do not believe we are harming the open platforms by developing on the iPhone. In fact, if all goes well, we will be allowing them to conquer the Apple iPhone.

Of course, I know the reply to all of this. "That sounds good, now show me the code." It's important not to overpromise and underdeliver, so I will be very cautious. What I have just said is the hope, the best possible outcome. But just having that as a possibility is tantalizing enough to justify working on this project. However, to be honest, my original justification (as stated to the dev team) for working on iPhone Linux was "for Skillz.app", our facetious term for working on something merely to hone one's skill or to satisfy one's curiosity. But honestly, what did you expect from a "hacker"? :)

We have already made more progress with openiboot than many people have anticipated would ever happen. Reverse engineering drivers is a laborious process, but one that doesn't require the luck of finding a security vulnerability: It just happens slowly and steadily, rather than unpredictably. Presumably after the drivers are in place, the Linux kernel will "just work" without too many other changes, since it is designed to be relatively portable, so we ought not to have many problems. After the kernel works, I hope enough developers will become interested and a nice userland can be developed without too much trouble. The userland work is much less risky from a time-investment point of view.

Boot Menus, So Far...

So, I'm already getting some boot menus from people! I'll put the entries up here. Please comment and critique them. I will also be giving some of my thoughts:

These two are from chris custren:

These two are from poorlad:

This one is from pH:

That's it so far. If you've got any comments or suggestions about them, leave them here. Here's what came up from a technical/practical perspective: As I've mentioned, gradients are good. It is also very good if you can do things in grayscale (as poorlad did for one of his images, but even the blue scale can be programmatically generated) since it saves on expensive bitmap data. On using the Apple logo, here's what we're willing to do. We're willing to accept a logo with an original, unaltered Apple logo OR a logo created from scratch that resembles the Apple trademark. We cannot accept logos that modifies copyrighted Apple materials. The rationale for using Apple trademarks is because this is merely for identification purposes. If their lawyers have any problems with that, we will respect any requests to take it down.

NAND FTL

So the big news yesterday (other than Obama winning the presidency!) is that we have enough of a low-level NAND driver now that we're able to read from NAND! It was epic win. There turns out to be not as much hardware voodoo as, say, Merlot, so that's pretty good news. It seems to work (albeit slowly) and I even wrote the ECC routines today (and those seem to work as well).

Unfortunately, in the course of this, we discovered several unfortunate things. First, I can't seem to find anything that might write to NAND. It's probably not much more complicated and probably reuses a lot of the stuff we've been doing, but it means that we might have to look in the kernel for that code, which sort of bites (a lot of the kernel is in C++ and not as friendly to reverse).

The second thing is the realization that all of Samsung's proprietary FTL code is in this thing. Without being able to understand it, we can't actually map sectors to data and we can't make sense of the NAND data or write new data to it in a useful way. Unfortunately, this code is liable to be ridiculously complex, since it's basically their SDK they ship to everyone. Without it, we can still proceed, but the iPhone can't read Linux's data and Linux can't read iPhone's data. In the worst case, we can't even have both OSes on the NAND at once.

Still, being able to dump NAND through USB is a substantial accomplishment, and we're well on our way.

Boot Menu Art

I think the best way to handle this is for anyone interested to submit a proposal via e-mail to me about the boot menu. Then, I can put up the pictures on the blog for people to comment and have an opinion about. We can then figure out which one to use or maybe some combination of proposals or in any case, figure it out in a collaborative community style process.

I don't really have an opinion on what the logo should look like, whether it should be cartoony or not. I only think it should look good. =P

There are very few technical constraints on the boot menu. I'm willing to figure out whatever technical solution there needs be in order to get things to display properly. One thing to keep in mind is that space on the NOR is at a significant premium, and raw pixel data is expensive but procedurally generated stuff like gradients are possible. The only two current menu options are the iPhone OS and the openiboot console, but eventually a Linux option will be added. I can try animation as well, but again, you'd have to have an idea of how it can be done without using up a lot of NOR space.

But yeah, make us pretty!

Some progress on the NAND: Thanks to a huge amount of initial grunt work by CPICH, the NAND project is off the ground. We already have enough to get the proper drive geometries, and I've written and tested the DMA routines that are near the heart of the problem today. Hopefully, we can dump the raw NAND soon and take a peek at what kind of wear leveling data structures we're dealing with.

Installation, the PMU

While I was waiting for CPICH to finish the first bits of the NAND FTL reverse engineering work, I've been trying to fill in some of the gaps we had in other places, such as the PMU. As promised, there is also now an easy way to install openiboot onto the iPhone. This is great because it will eventually lead to an even leaner and easier QuickPwn in the future.

One of the annoying parts about iBoot in recovery mode is that the thing refuses to charge the iPhone while sitting in recovery mode. The battery just eventually entirely drains. With the new PMU code, openiboot now recharges the battery, so programmers using it (read: me) can just have it sit on the console screen indefinitely. You can also do neat things like check the current battery voltage and check the power supply type the phone is charging from.

The "installation code" consists of porting over my knowledge of reading and modifying img3 files from working on the jailbreaks. I was too lazy to port over the entire xpwn framework, but I wrote up a "diet" version that is sufficient to read and modify img3 files in a limited fashion. img3 files are sort of the new native format of the main part of the NOR (just a bunch of img3 files concatenated together). The upshot is that you can load openiboot as an img3 through iBoot (just like sending an iBEC image) and then type "install" at the console and openiboot will be a permanent stage in your bootloader chain. =P

You can, of course, keep booting up to the iPhone OS as you always do by selecting the option in the boot menu. Installing openiboot isn't very useful except for hackers wanting to hack openiboot.

I also figured out how to parse and modify the NVRAM banks (storing environment variables like "auto-boot", etc.), which was actually pointless complicated (in my opinion). They have two banks consisting of a bunch of partitions with these headers that Apple uses a pointless one-byte custom checksum on. The entire bank is also checksumed with adler32. When NVRAM is modified, the oldest bank is overwritten with the data and becomes the newest bank (which is tracked by an epoch number on each bank). This is so if one bank becomes corrupted, the other can be used as a backup. However, NVRAM hardly contains anything high value so the value of all this trouble is doubtful. Being able to write to NVRAM, though, makes it possible to set auto-boot on and off within openiboot so that we can easily control whether or not to enter iBoot's recovery mode.

Someone asked me how "safe" it was to do the installation, etc. Well, I've been doing it every time I make an update these days, so it's fairly safe. The worst that can happen in the usual case is that you may be forced into a DFU mode restore. Everything will be undone with a restore. Early on, I did have bugs that really screwed things up so that a DFU mode restore was no longer possible, but even that was recoverable. I'll just go over how briefly:

The important thing is to have a backup of the NOR. As I described in a previous posting, it's possible to really screw things up if you erase the SysCfg section of the NOR. If you do that, the iPhone OS will refuse to boot at all since iBoot cannot properly populate the device tree for the kernel. Since restore ramdisks rely on XNU booting, this is Bad News Bears. In addition, the SysCfg section is device specific, so if you do not have a backup, it will be difficult to ever completely recover from erasing it.

Therefore, before you proceed, MAKE A BACKUP OF YOUR NOR. openiboot can do this for you (and subsequently restore your backup if things go wrong).

Load openiboot via loadibec and select the console. Connect with the oibc client. Type in: nor_read 0x09000000 0x0 0x100000

This will read all of NOR into memory. Then type: ~nordump.bin:0x100000

This will transfer the dump over USB onto your computer and save it as nordump.bin.

Supposing you filled the entire NOR with garbage somehow and are unable to boot. You have to get into openiboot to restore the NOR. The problem is that openiboot is only designed to operate in a post-LLB or post-Recovery Mode context, so it cannot be directly booted from DFU mode. Basically, you've got to load a pwned WTF, then a pwned iBSS, and then a pwned iBEC (all of which is available from a custom IPSW). After that, you can use loadibec to load openiboot. Then, you can restore the NOR thus:

!nordump.bin
nor_write 0x09000000 0x0 0x100000

After that, you can reboot and everything should be normal.

Also, I received a few responses for people volunteering to do the art. I'm not sure what the best thing would be, since I don't want anyone putting in effort for nothing, but we do want the best possible results. So, I'll be getting back to you guys about that.