Flash Layout/Flash 布局

Flash Layout/Flash 布局

by https://openwrt.org/zh/docs/techref/flash.layout

有种东西叫做硬盘,被认为是 块(block)设备, 与之类似的是闪存(flash memory). Flash 的类型有很多, NOR, SLC NAND, MLC NAND等

如果闪存芯片是直接连接到CPU(SoC),并且能够直接被操作系统/Linux读取, 我们就把他叫做“raw flash”.如果闪存芯片不能够直接被操作系统/Linux读取 (由于闪存和SoC之间还需要另外的控制芯片来连接), 我们叫他 “FTL (Flash Translation Layer) flash”. 嵌入式系统几乎全部使用“raw flash”, 而USB 闪存盘/U盘/SSD固态硬盘, 几乎全部使用 FTL flash!

SquashFS文件的分区

嵌入式系统中的Flash芯片并不需要单独的控制芯片,因此不是 “FTL”-设备 而是 “raw flash”-设备. 存储空间是通过操作MTD设备 MTD (Memory Technology Device)加上特定的文件系统filesystems来完成的. 传统的存储空间使用 MBR 和 PBRs 来存储分区相关的信息, 而嵌入式设备中 这由内核 Linux Kernel 来完成(而且有时候会单独由bootloader来完成!). 你只需要定义, “kernel分区 由偏移量x起 至偏移量y止”. 这样做的优点是, 我们之后能直接通过 分区名来定位某个分区,而不用指定数据的精确起始点.
嵌入式系统中的Flash芯片并不需要单独的控制芯片,因此不是“FTL”-设备 而是“raw flash”-设备。存储空间是通过操作MTD设备MTD(Memory Technology Device)附加特定的文件系统文件系统完成的传统的存储空间使用MBR和PBR来存储独立分区相关的信息,而嵌入式设备中 这由内核 Linux Kernel 来完成(而且有时会单独由bootloader 来完成!)你只需要定义,“内核区由偏移量x起 至偏移量y止”。这样做的优点是,我们可以在之后直接通过 分区命名来定位某个分区,而不必使用指定数据的精确起始点。

下图中的表格是 TP-Link TL-WR1043ND型号的分区图, 注意这只是一个例子! 另一个, 与这个有些微不同的, 例子能在wiki 的这个页面找到 – DIR-300. 注意不同的设备的Flash布局是不同的! 请参考wiki页面上针对特定设备的页面, 来获取该设备的Flash布局信息.
In this paper,we discuss the relationship between thetransmitting and receiving of the transmitting signals and the transmitting signals.另一个,与这个有些微不同的,例子能在维基的这个界面找到-Note that Flash layout is different from the different device!请参考wiki界面上针对特定设备的界面,来获取该设备的Flash布局信息。

TP-Link WR1043ND Flash Layout TP-Link WR 1043 ND闪存布局
Layer0 层0
Layer1
Layer2
mountpoint
filesystem 文件系统
Layer3
Size in KiB 大小为KiB
Name 名称
mountpoint
filesystem 文件系统

分区

由于分区是嵌套(nested)的,我们把他分成几层来看:

  1. 第0层Layer0: 对应Flash芯片,8MiB大小, 焊接在PCB上,连接到CPU(SoC)socSoC – 通过[SPI (Serial Peripheral Interface Bus)串行外设接口总线](https://en.wikipedia.org/wiki/Serial Peripheral Interface Bus “https://en.wikipedia.org/wiki/Serial Peripheral Interface Bus”)总线.
  2. 第1层Layer1: 我们把存储空间“分区”为 mtd0 给 bootloader, mtd5 给 firmware/固件使用, 并且, 在这个例子中, mtd4给ART (Atheros Radio Test/Atheros电波测试) - 它包含MAC地址和无线系统的校准信息(EEPROM). 如果该部分的信息丢失或损坏,ath9k (无线驱动程序) 就彻底罢工了.
  3. 第2层Layer2: 我们把mtd5 (固件) 进一步分割为 mtd1 (kernel/内核) and mtd2 (rootfs); 在固件的一般处理流程中 (参考imagebuilder图像生成器) Kernel 二进制文件 先由[LZMAlzma](https://en.wikipedia.org/wiki/Lempel–Ziv–Markov chain algorithm “https://en.wikipedia.org/wiki/Lempel–Ziv–Markov chain algorithm”)打包, 然后用gzip压缩 之后文件被 直接写入到raw flash (mtd1)中 而不mount到任何文件系统上!
  4. 第3层Layer3: 我们把rootfs更进一步分割成 mtd3 (rootfs_data) 和剩下的 未命名 部分 - 未来用来容纳SquashFS-分区.

文件系统

→ 参考文件系统专题页面filesystems
→ 参考文件系统专题界面文件系统

挂载点

  • / 这是文件系统根入口,它由 /rom/overlay构成.。在日常使用中请忽略 /rom/overlay目录!
  • /rom 包含所有基础文件,比如 busybox, dropbear 也或 iptables包含的缺省配置文件。此目录不包含内核,此目录重点文件是在SqashFS区域, 因此文件无法删除。但是,因为我们使用的是mini_fo文件系统, 所谓的 overlay-whiteout覆盖白化 -symlinks可以被创建在JFFS2区域.
  • /overlay/rom共同构成统一的 /根目录,/overlay是可读可写的部分。它包含It contains anything that was written to the router after installation安装, e.g. changed configuration files, additional packages installed with ,例如更改配置文件,安装附加软件包, <a href="https://openwrt.org/docs/guide-user/additional-software/opkg" class="wikilink1" title="docs:guide-user:additional-software:opkg" data-wiki-id="docs:guide-user:additional-software:opkg">opkg</a>, etc. It is formated with JFFS2.它是用JFFS2格式的。

Rather than deleting the files, insert a whiteout, a special high-priority entry that marks the file as deleted. File system code that sees a whiteout entry for file F behaves as if F does not exist.
而不是删除文件,插入一个whiteout,一个特殊的高优先级条目,将文件标记为已删除。看到文件F的空白项的文件系统代码表现得好像F不存在。

#!/bin/bash
# shows all overlay-whiteout symlinks
 
find /overlay -type l | while read FILE
  do
    [ -z "$FILE" ] && break
    if ls -la "$FILE" 2>&- | grep -q '(overlay-whiteout)'; then
    echo "$FILE"
    fi
  done

目录

NOTE1: If the Kernel was part of the SquashFS, we could not control where exactly on the flash it is written to (on which blocks it’s data is contained). Thus we could not tell the bootloader to simply load and execute certain blocks on the flash storage, but would have to address it with path and filename. Now this would not be bad, but in order to that the bootloader would have to understand the SquashFS filesystem, which it does not. The embedded bootloader we utilize with OpenWrt generally have no concept of filesystems, thus they cannot address files by path and filename. They pretty much assume that the start of the trx data section is executable code.NOTE2: the denomination “firmware” usually is used for the entire data on the flash comprising the boot loader and any other data necessary to operate the device, such as ART, NVRAM, FIS, etc, but we also use it to only name the parts that are being rewritten. Don’t let this confuse you
NOTE1:如果内核是SquashFS的一部分,我们无法控制它在闪存上的确切位置(它的数据包含在哪个块上)。因此,我们不能告诉引导程序简单地加载和执行闪存上的某些块,而是必须用路径和文件名来寻址它。这并不坏,但是为了做到这一点,引导加载程序必须理解SquashFS文件系统,而它并不理解。我们在OpenCart中使用的嵌入式引导程序通常没有文件系统的概念,因此它们不能通过路径和文件名来寻址文件。他们几乎假定trx数据段的开始是可执行代码。二号:命名NOTE2通常用于闪存上的整个数据,包括靴子加载程序和操作设备所需的任何其他数据,例如ART、NVRAM、FIS等,但我们也使用它来仅命名正在重写的部分。别让这件事迷惑了你

JFFS2-Images 分区JFFS 2-Images分区

TODOtodo

Discovery (How to find out)发现(How to find out)

cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00020000 00010000 "u-boot"
mtd1: 00140000 00010000 "kernel"
mtd2: 00690000 00010000 "rootfs"
mtd3: 00530000 00010000 "rootfs_data"
mtd4: 00010000 00010000 "art"
mtd5: 007d0000 00010000 "firmware"

The erasesize is the block size of the flash, in this case 64KiB. The size is little or big endian hex value in Bytes. In case of little endian, you switch to hex-mode and enter 02 0000 into the calculator for example and convert to decimal (by switching back to decimal mode again). Then guess how they are nested into each other. Or execute dmesg after a fresh boot and look for something like:
erasesize是闪存的块大小,在本例中为64KiB。大小是十六进制的小或大端值。在小端的情况下,你切换到十六进制模式,并输入02 0000到计算器,例如,并转换为十进制(通过再次切换回十进制模式)。然后猜猜他们是如何互相嵌套的。或者在新的靴子后执行dmesg,并寻找类似的东西:

Creating 5 MTD partitions on "spi0.0":
0x000000000000-0x000000020000 : "u-boot"
0x000000020000-0x000000160000 : "kernel"
0x000000160000-0x0000007f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=2C0000, len=530000
0x0000002c0000-0x0000007f0000 : "rootfs_data"
0x0000007f0000-0x000000800000 : "art"
0x000000020000-0x0000007f0000 : "firmware"

These are the start and end offsets of the partitions as hex values in Bytes. Now you don’t have to guess which is nested in which. E.g. 02 0000 = 131.072 Bytes = 128KiB.
这些是分区的开始和结束偏移量,以十六进制值表示。现在你不必猜哪个嵌套在哪个里面了。例如,02 0000 = 131.072 kB = 128 KiB。

Details细节

generic通用

The flash chip can be represented as a large block of continuous space:
闪存芯片可以表示为一大块连续空间:

start of flash …………….. end of flash 闪光开始…..闪光结束

There is no ROM to boot from; at power up the CPU begins executing the code at the very start of the flash. Luckily this isn’t the firmware or we’d be in real danger every time we reflashed. Boot is actually handled by a section of code we tend to refer to as the bootloader (the BIOS of your PC is a bootloader).
没有ROM可供靴子;在上电时,CPU在闪存的最开始就开始执行代码。幸运的是,这不是固件,否则我们将在真实的危险,每次我们重新启动。靴子实际上是由一段代码处理的,我们倾向于将其称为引导加载程序(您PC的BIOS就是一个引导加载程序)。

Boot Loader Partition 靴子加载程序分区 Firmware Partition 分区Firmware Partition Special Configuration Data
Atheros U-Boot`` U—Boot firmware 固件 ART
Broadcom CFE firmware 固件 NVRAM
Atheros RedBoot redboot firmware 固件 FIS recovery

The partition or partitions containing so called Special Configuration Data differ very much from each other. Example: The ART-partition you will meet in conjunction with Atheros-Wireless and U-Boot, contains only data regarding the wireless driver, while the NVRAM-partition of broadcom devices is used for much more then only that.
包含所谓的特殊配置数据的分区或分区彼此差异很大。例如:您将与Atheros—Wireless和U—Boot一起遇到的ART分区仅包含有关无线驱动程序的数据,而Broadcom设备的NVRAM分区则用于更多功能。

broadcom with CFEBroadcom与CFE

If you dig into the “firmware” section you’ll find a trx. A trx is just an encapsulation, which looks something like this:
如果你深入到“固件”部分,你会发现一个trx。trx只是一个封装,看起来像这样:

trx-header trx报头
HDR0 人类发展报告0

“HDR0” is a magic value to indicate a trx header, rest is 4 byte unsigned values followed by the actual contents. In short, it’s a block of data with a length and a checksum. So, our flash usage actually looks something like this:
“HDR0”是一个魔术值,表示一个trx头,其余是4字节的无符号值,后面是实际内容。简而言之,它是一个具有长度和校验和的数据块。所以,我们的Flash使用实际上看起来像这样:

CFE trx containing firmware 包含固件的trx NVRAM

Except that the firmware is generally pretty small and doesn’t use the entire space between CFE and NVRAM:
除了固件通常非常小,并且不会使用CFE和NVRAM之间的整个空间:

CFE trx firmware trx固件 unused 未使用 NVRAM

(NOTE: The <model>.bin files are nothing more than the generic *.trx file with an additional header appended to the start to identify the model. The model information gets verified by the vendor’s upgrade utilities and only the remaining data – the trx – gets written to the flash. When upgrading from within OpenWrt remember to use the *.trx file.)
(NOTE:model.bin文件只不过是一个通用的 *.trx文件,在文件的开头附加了一个额外的头文件来标识模型。型号信息由供应商的升级实用程序验证,只有剩余的数据(trx)写入闪存。当从OpenSSL内部升级时,请记住使用 *.trx文件。)

So what exactly is the firmware?那么到底什么是固件?

The boot loader really has no concept of filesystems, it pretty much assumes that the start of the trx data section is executable code. So, at the very start of our firmware is the kernel. But just putting a kernel directly onto flash is quite boring and consumes a lot of space, so we compress the kernel with a heavy compression known as LZMA. Now the start of firmware is code for an LZMA decompress:
靴子loader真的没有文件系统的概念,它几乎假设trx数据段的开始是可执行代码。因此,在我们的固件的最开始是内核。但是直接将内核放在闪存上是非常无聊的,并且会消耗大量空间,因此我们使用称为LZMA的重压缩来压缩内核。现在,固件的开始是LZMA固件的代码:

lzma decompress 伊兹马河 lzma compreszsed kernel lzma压缩核

Now, the boot loader boots into an LZMA program which decompresses the kernel into memory and executes it. It adds one second to the bootup time, but it saves a large chunk of flash space. (And if that wasn’t amusing enough, it turns out the boot loader does know gzip compression, so we gzip compressed the LZMA decompression program)
现在,靴子loader引导进入一个LZMA程序,该程序将内核解压缩到内存中并执行它。(And如果这还不够有趣的话,原来靴子加载程序确实知道gzip压缩,所以我们用gzip压缩了LZMA解压缩程序)

Immediately following the kernel is the filesystem. We use SquashFS for this because it’s a highly compressed readonly filesystem – remember that altering the contents of the trx in any way would invalidate the crc, so we put our writable data in a JFFS2 partition, which is outside the trx. This means that our firmware looks like this:
紧随内核之后的是文件系统。我们使用SquashFS,因为它是一个高度压缩的只读文件系统–记住,以任何方式更改trx的内容都会使crc无效,所以我们将可写数据放在一个JFFS 2分区中,该分区在trx之外。这意味着我们的固件看起来像这样:

trx TRX gzip’d lzma decompress 伊兹玛角 lzma’d kernel lzma’d内核 (SquashFS filesystem) (SquashFS文件系统)

And the entire flash usage looks like this -
整个flash的使用情况是这样的-

CFE trx TRX gz’d lzma 格兹伊兹马 lzma’d kernel lzma’d内核 SquashFS JFFS2 filesystem JFFS2文件系统 NVRAM

That’s about as tight as we can possibly pack things into flash.
这是我们能把东西装进闪光灯的最小限度了。


Explanations解释

What is an Image File?什么是图像文件?

An image file is byte by byte copy of data contained in a file system. If you installed a Debian or a Windows in the usual way onto one or two harddisc partitions and would afterwards copy the whole content byte by byte from the hard disc into one file:
映像文件是文件系统中包含的数据的逐字节副本。如果您以通常的方式将Debian或Windows安装到一个或两个硬盘分区上,然后将整个内容从硬盘中逐字节复制到一个文件中:

dd if=/dev/sda of=/media/sdb3/backup.dd

the obtained backup file /media/sdb3/backup.dd, could be used in the exact same manner like an OpenWrt-Image-File.
所获得的备份文件/media/sdb3/backup.dd可以以与OpenBit映像文件完全相同的方式使用。

The difference is, that OpenWrt-Image-File are not created that way They are being generated with the Image Generator (former called Image Builder). You can read about the:
不同的是,OpenCart图像文件不是这样创建的,它们是用图像生成器(以前称为图像生成器)生成的。您可以阅读以下内容:


转载请注明来源,欢迎对文章中的引用来源进行考证,欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论,也可以邮件至 3400639399@qq.com

×

喜欢就点赞,疼爱就打赏

github