Parsing/converting Nokia "Smart Feature OS" backup .ib files?

I have already written a bit on this in https://superuser.com/questions/1389657/backup-access-sms-on-nokia-3310-3g-2017-from-linux-pc ; basically, I'm trying to back up SMS messages on a Nokia 3310 3G onto a Ubuntu 18.04 PC; note that hardware system-on-chip and OS differs by version for a Nokia 3310 (2017):

System on chip / Operating system:

MediaTek MT6260 / Nokia Series 30+ (2G)

Spreadtrum SC7701B / Java-powered Smart Feature OS (3G)

Spreadtrum SC9820A / Yun OS (4G, CMCC)

I have the 3G, so I have a "Smart Feature OS" which apparently is a version of KaiOS (Is there any difference between KaiOS and 'Smart Feature OS'? : KaiOS), which apparently (KaiOS – A Smartphone Operating System | Hacker News) is a fork of Firefox OS.

When you hit Menu > Storage > Create backup (https://www.nokia.com/phones/en_int/support/nokia-3310-3g-user-guide/create-a-backup) on this phone, it generates a folder with files in it, named like this:

$ tree All-backup_01-01-2019_20-18-54
All-backup_01-01-2019_20-18-54
├── ibphone_head.in
├── phonebook.ib
└── sms.ib

I've never heard of .ib files before, and I was hoping someone here knew what they are. A quick look suggests IB File Extension - Open .IB File (InterBase Database), however I tried using these .ib files with http://fbexport.sourceforge.net/fbexport.php "Tool for exporting and importing data with Firebird and InterBase databases", and I get:

Engine Code    : 335544323
Engine Message :
file ./All-backup_01-01-2019_20-18-54/phonebook.ib is not a valid database

So, that's not it.

Here is a hexdump of ibphone_head.in - looks like there is no personal identifying info here:

$ hexdump -C All-backup_01-01-2019_20-18-54/ibphone_head.in 
00000000  00 00 00 00 41 00 6c 00  6c 00 2d 00 62 00 61 00  |....A.l.l.-.b.a.|
00000010  63 00 6b 00 75 00 70 00  5f 00 30 00 31 00 2d 00  |c.k.u.p._.0.1.-.|
00000020  30 00 31 00 2d 00 32 00  30 00 31 00 39 00 5f 00  |0.1.-.2.0.1.9._.|
00000030  32 00 30 00 2d 00 31 00  38 00 2d 00 35 00 34 00  |2.0.-.1.8.-.5.4.|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  00 00 00 00 45 00 3a 00  5c 00 42 00 61 00 63 00  |....E.:.\.B.a.c.|
00000110  6b 00 75 00 70 00 73 00  5c 00 41 00 6c 00 6c 00  |k.u.p.s.\.A.l.l.|
00000120  2d 00 62 00 61 00 63 00  6b 00 75 00 70 00 5f 00  |-.b.a.c.k.u.p._.|
00000130  30 00 31 00 2d 00 30 00  31 00 2d 00 32 00 30 00  |0.1.-.0.1.-.2.0.|
00000140  31 00 39 00 5f 00 32 00  30 00 2d 00 31 00 38 00  |1.9._.2.0.-.1.8.|
00000150  2d 00 35 00 34 00 00 00  00 00 00 00 00 00 00 00  |-.5.4...........|
00000160  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  00 00 00 00 30 30 30 31  2e 30 30 30 30 33 00 00  |....0001.00003..|
00000210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000230  00 00 00 00 00 00 6d 6d  69 6b 65 79 62 61 63 6b  |......mmikeyback|
00000240  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000250  00 00 00 00 00 00 03 00  00 00 00 00 b0 cd 09 00  |................|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000270  00 00 00 00 00 00 00 00  f3 dd 00 00 7e 2f 00 00  |............~/..|
00000280  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000002c4

So, it seems that strings are encoded with 2 bytes, "wide char"; and for the most part, ibphone_head.in seems to just encode the name of its containing/parent folder, All-backup_01-01-2019_20-18-54.

Here is a hexdump of phone book, where I've anonymized names to AAAAAAAAA and BBB:

$ hexdump -C -n 1900 All-backup_01-01-2019_20-18-54/phonebook.ib
00000000  70 00 68 00 6f 00 6e 00  65 00 62 00 6f 00 6f 00  |p.h.o.n.e.b.o.o.|
00000010  6b 00 2e 00 69 00 62 00  00 00 00 00 00 00 00 00  |k...i.b.........|
00000020  00 00 00 00 01 00 00 00  30 f8 04 00 62 01 00 00  |........0...b...|
00000030  62 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |b...............|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000240  00 00 00 00 98 03 00 00  01 00 00 00 ff ff ff ff  |................|
00000250  ff ff ff ff 01 00 01 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000360  d9 d4 37 46 00 00 00 00  00 00 01 02 00 00 04 01  |..7F............|
00000370  07 12 80 88 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000380  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000003b0  09 00 41 00 41 00 41 00  41 00 41 00 41 00 41 00  |..A.A.A.A.A.A.A.|
000003c0  41 00 41 00 00 00 00 00  00 00 00 00 00 00 00 00  |A.A.............|
000003d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000005e0  00 00 00 00 00 00 00 00  00 00 00 00 ff ff ff ff  |................|
000005f0  ff ff ff ff 98 03 00 00  01 00 00 00 ff ff ff ff  |................|
00000600  ff ff ff ff 04 00 01 00  00 00 00 00 00 00 00 00  |................|
00000610  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000710  d9 d4 37 46 00 00 00 00  00 00 01 02 00 00 06 11  |..7F............|
00000720  83 29 23 13 58 f9 00 00  00 00 00 00 00 00 00 00  |.)#.X...........|
00000730  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000760  03 00 42 00 42 00 42 00  00 00 00 00              |..B.B.B.....|
0000076c

Here it seems d9 d4 37 46 is a delimiter for entries, and entries seem to be 0x3b0 = 944 bytes apart; cannot tell where the actual phone number is stored, though.

I won't be pasting the hexdump contents of sms.ib, because I fear to reveal more personal info than I intend to.

However, maybe what has been posted already, will help someone see if this is an already established file format? In any case, I'd like to convert the contents of these files to plain text ...

Solution

The phonebook.ib is a proprietary file format. I did some analysis and got to the point where I am able to extract names and phones numbers from it.

Header is 580 bytes and doesn't seem to contain anything interesting.
Entries appear to begin with the entry length, encoded as 3 4-bit decimal nibbles followed by another digit (version number maybe?).

In my sample file all entries were 940 bytes and therefore had 94 03 as their first two bytes. Other field entries I identified at different offsets:

entry+0x12a [1 byte]: Number of bytes representing phone number.
entry+0x12b [1 byte]: Seems like a two-digit decimal coded flag. If high nibble is set (e.g. 10) then phone number begins with +.
entry+0x12c: Phone number, decimal coded. For example, 123456 would appear as 21 43 65. Special digits:
- a is *
- b is #
- f is ignored (seen as a last digit when the number of digits is not even).
entry+0x16c [1 byte]: Name length.
entry+0x16e: Name in UTF-16 (i.e. 2 bytes per char).

Taking your snippet as an example:

0x36e is the phone length, 4 bytes.
0x36f is the extra flag, high nibble is 0 so no + prefix.
0x370 is where the phone number begins, 07 12 80 88 translates to 70210888.

A simple reference parser can be found at my repo: https://github.com/yossigo/phonebook_ib_export