docker ビルド時のレイヤー上限は 125 (暫定) という話。終わり。
動作確認
OS は Ubuntu 、docker は 20.10.2 を利用する。
# docker --version
Docker version 20.10.2, build 20.10.2-0ubuntu1~20.04.2
以下のようなファイルを用意してイメージをビルドする。
FROM alpine:latest
WORKDIR /app
RUN touch 1.txt
RUN touch 2.txt
RUN touch 3.txt
...
すると、Step 126 で max depth exceeded でエラーになる。
# docker build -t too-many-layers -f Dockerfile .
Sending build context to Docker daemon 6.144kB
Step 1/127 : FROM alpine:latest
---> 6dbb9cc54074
Step 2/127 : WORKDIR /app
---> Running in 035bdb197382
Removing intermediate container 035bdb197382
---> 14ab0955f1a6
Step 3/127 : RUN touch 1.txt
---> Running in db3ece39cf58
...
Step 126/127 : RUN touch 124.txt
---> Running in a192710bcc70
max depth exceeded
なんで？
以下のような Issue があった。
The maximum number of layers is undocumented. · Issue #8230 · docker/docker.github.iogithub.com
実装上は次のコードでエラーを出していた。 moby/moby layer/layer_store.go
if p.depth() >= maxLayerDepth {
err = ErrMaxDepthExceeded
return nil, err
}
コメントを見ると、graphdrivers の上限が 127 なので rwlayer のために 2 減らして 125 としているらしい。
// maxLayerDepth represents the maximum number of
// layers which can be chained together. 125 was
// chosen to account for the 127 max in some
// graphdrivers plus the 2 additional layers
// used to create a rwlayer.
const maxLayerDepth = 125
いくつかの graphdriver ではコンテナ実行時にもレイヤー数をチェックしており、これも同様の上限になっていた。 moby/moby daemon/graphdriver/overlay2/overlay.go
こちらの記載のほうが分かりやすいが、 mount コマンドの上限による都合の様子。
maxDepth = 128
// idLength represents the number of random characters
// which can be used to create the unique link identifier
// for every layer. If this value is too long then the
// page size limit for the mount command may be exceeded.
// The idLength should be selected such that following equation
// is true (512 is a buffer for label metadata).
// ((idLength + len(linkDir) + 1) * maxDepth) <= (pageSize - 512)
試しに overlayfs でたくさんのレイヤーを重ねてみたが、ある程度の長さで mount コマンドが失敗している。
# mkdir layers merged overlay upper work
# seq 1 1000 | xargs -I @@ bash -c "mkdir layers/@@ ; touch layers/@@/@@.txt"
# mount -t overlay overlay -o lowerdir=$(seq 1 100 | sed "s#^#layers/#g" | tr "\n" ":" | sed "s/:$//g"),upperdir=upper,workdir=work merged
# ls merged/
100.txt 13.txt 17.txt 20.txt 24.txt 28.txt 31.txt 35.txt 39.txt 42.txt 46.txt 4.txt 53.txt 57.txt 60.txt 64.txt 68.txt 71.txt 75.txt 79.txt 82.txt 86.txt 8.txt 93.txt 97.txt
10.txt 14.txt 18.txt 21.txt 25.txt 29.txt 32.txt 36.txt 3.txt 43.txt 47.txt 50.txt 54.txt 58.txt 61.txt 65.txt 69.txt 72.txt 76.txt 7.txt 83.txt 87.txt 90.txt 94.txt 98.txt
11.txt 15.txt 19.txt 22.txt 26.txt 2.txt 33.txt 37.txt 40.txt 44.txt 48.txt 51.txt 55.txt 59.txt 62.txt 66.txt 6.txt 73.txt 77.txt 80.txt 84.txt 88.txt 91.txt 95.txt 99.txt
12.txt 16.txt 1.txt 23.txt 27.txt 30.txt 34.txt 38.txt 41.txt 45.txt 49.txt 52.txt 56.txt 5.txt 63.txt 67.txt 70.txt 74.txt 78.txt 81.txt 85.txt 89.txt 92.txt 96.txt 9.txt
# umount merged
# mount -t overlay overlay -o lowerdir=$(seq 1 1000 | sed "s#^#layers/#g" | tr "\n" ":" | sed "s/:$//g"),upperdir=upper,workdir=work merged
mount: /tmp/fs/merged: mount(2) system call failed: No such file or directory.
mount コマンドに渡すまえに bash でエラーになってなりしないか確認したが、strace コマンドの結果から引数は正常そうなことがわかる。そのため、mount システムコール内で失敗している思われる。
# strace -s 1000000 -e mount mount -t overlay overlay -o lowerdir=$(seq 1 1000 | sed "s#^#layers/#g" | tr "\n" ":" | sed "s/:$//g"),upperdir=upper,workdir=work merged
mount("overlay", "/tmp/fs/merged", "overlay", 0, "lowerdir=layers/1:layers/2:layers/3:layers/4:layers/5:layers/6:layers/7:layers/8:layers/9:layers/10:layers/11:layers/12:layers/13:layers/14:layers/15:layers/16:layers/17:layers/18:layers/19:layers/20:layers/21:layers/22:layers/23:layers/24:layers/25:layers/26:layers/27:layers/28:layers/29:layers/30:layers/31:layers/32:layers/33:layers/34:layers/35:layers/36:layers/37:layers/38:layers/39:layers/40:layers/41:layers/42:layers/43:layers/44:layers/45:layers/46:layers/47:layers/48:layers/49:layers/50:layers/51:layers/52:layers/53:layers/54:layers/55:layers/56:layers/57:layers/58:layers/59:layers/60:layers/61:layers/62:layers/63:layers/64:layers/65:layers/66:layers/67:layers/68:layers/69:layers/70:layers/71:layers/72:layers/73:layers/74:layers/75:layers/76:layers/77:layers/78:layers/79:layers/80:layers/81:layers/82:layers/83:layers/84:layers/85:layers/86:layers/87:layers/88:layers/89:layers/90:layers/91:layers/92:layers/93:layers/94:layers/95:layers/96:layers/97:layers/98:layers/99:layers/100:layers/101:layers/102:layers/103:layers/104:layers/105:layers/106:layers/107:layers/108:layers/109:layers/110:layers/111:layers/112:layers/113:layers/114:layers/115:layers/116:layers/117:layers/118:layers/119:layers/120:layers/121:layers/122:layers/123:layers/124:layers/125:layers/126:layers/127:layers/128:layers/129:layers/130:layers/131:layers/132:layers/133:layers/134:layers/135:layers/136:layers/137:layers/138:layers/139:layers/140:layers/141:layers/142:layers/143:layers/144:layers/145:layers/146:layers/147:layers/148:layers/149:layers/150:layers/151:layers/152:layers/153:layers/154:layers/155:layers/156:layers/157:layers/158:layers/159:layers/160:layers/161:layers/162:layers/163:layers/164:layers/165:layers/166:layers/167:layers/168:layers/169:layers/170:layers/171:layers/172:layers/173:layers/174:layers/175:layers/176:layers/177:layers/178:layers/179:layers/180:layers/181:layers/182:layers/183:layers/184:layers/185:layers/186:layers/187:layers/188:layers/189:layers/190:layers/191:layers/192:layers/193:layers/194:layers/195:layers/196:layers/197:layers/198:layers/199:layers/200:layers/201:layers/202:layers/203:layers/204:layers/205:layers/206:layers/207:layers/208:layers/209:layers/210:layers/211:layers/212:layers/213:layers/214:layers/215:layers/216:layers/217:layers/218:layers/219:layers/220:layers/221:layers/222:layers/223:layers/224:layers/225:layers/226:layers/227:layers/228:layers/229:layers/230:layers/231:layers/232:layers/233:layers/234:layers/235:layers/236:layers/237:layers/238:layers/239:layers/240:layers/241:layers/242:layers/243:layers/244:layers/245:layers/246:layers/247:layers/248:layers/249:layers/250:layers/251:layers/252:layers/253:layers/254:layers/255:layers/256:layers/257:layers/258:layers/259:layers/260:layers/261:layers/262:layers/263:layers/264:layers/265:layers/266:layers/267:layers/268:layers/269:layers/270:layers/271:layers/272:layers/273:layers/274:layers/275:layers/276:layers/277:layers/278:layers/279:layers/280:layers/281:layers/282:layers/283:layers/284:layers/285:layers/286:layers/287:layers/288:layers/289:layers/290:layers/291:layers/292:layers/293:layers/294:layers/295:layers/296:layers/297:layers/298:layers/299:layers/300:layers/301:layers/302:layers/303:layers/304:layers/305:layers/306:layers/307:layers/308:layers/309:layers/310:layers/311:layers/312:layers/313:layers/314:layers/315:layers/316:layers/317:layers/318:layers/319:layers/320:layers/321:layers/322:layers/323:layers/324:layers/325:layers/326:layers/327:layers/328:layers/329:layers/330:layers/331:layers/332:layers/333:layers/334:layers/335:layers/336:layers/337:layers/338:layers/339:layers/340:layers/341:layers/342:layers/343:layers/344:layers/345:layers/346:layers/347:layers/348:layers/349:layers/350:layers/351:layers/352:layers/353:layers/354:layers/355:layers/356:layers/357:layers/358:layers/359:layers/360:layers/361:layers/362:layers/363:layers/364:layers/365:layers/366:layers/367:layers/368:layers/369:layers/370:layers/371:layers/372:layers/373:layers/374:layers/375:layers/376:layers/377:layers/378:layers/379:layers/380:layers/381:layers/382:layers/383:layers/384:layers/385:layers/386:layers/387:layers/388:layers/389:layers/390:layers/391:layers/392:layers/393:layers/394:layers/395:layers/396:layers/397:layers/398:layers/399:layers/400:layers/401:layers/402:layers/403:layers/404:layers/405:layers/406:layers/407:layers/408:layers/409:layers/410:layers/411:layers/412:layers/413:layers/414:layers/415:layers/416:layers/417:layers/418:layers/419:layers/420:layers/421:layers/422:layers/423:layers/424:layers/425:layers/426:layers/427:layers/428:layers/429:layers/430:layers/431:layers/432:layers/433:layers/434:layers/435:layers/436:layers/437:layers/438:layers/439:layers/440:layers/441:layers/442:layers/443:layers/444:layers/445:layers/446:layers/447:layers/448:layers/449:layers/450:layers/451:layers/452:layers/453:layers/454:layers/455:layers/456:layers/457:layers/458:layers/459:layers/460:layers/461:layers/462:layers/463:layers/464:layers/465:layers/466:layers/467:layers/468:layers/469:layers/470:layers/471:layers/472:layers/473:layers/474:layers/475:layers/476:layers/477:layers/478:layers/479:layers/480:layers/481:layers/482:layers/483:layers/484:layers/485:layers/486:layers/487:layers/488:layers/489:layers/490:layers/491:layers/492:layers/493:layers/494:layers/495:layers/496:layers/497:layers/498:layers/499:layers/500:layers/501:layers/502:layers/503:layers/504:layers/505:layers/506:layers/507:layers/508:layers/509:layers/510:layers/511:layers/512:layers/513:layers/514:layers/515:layers/516:layers/517:layers/518:layers/519:layers/520:layers/521:layers/522:layers/523:layers/524:layers/525:layers/526:layers/527:layers/528:layers/529:layers/530:layers/531:layers/532:layers/533:layers/534:layers/535:layers/536:layers/537:layers/538:layers/539:layers/540:layers/541:layers/542:layers/543:layers/544:layers/545:layers/546:layers/547:layers/548:layers/549:layers/550:layers/551:layers/552:layers/553:layers/554:layers/555:layers/556:layers/557:layers/558:layers/559:layers/560:layers/561:layers/562:layers/563:layers/564:layers/565:layers/566:layers/567:layers/568:layers/569:layers/570:layers/571:layers/572:layers/573:layers/574:layers/575:layers/576:layers/577:layers/578:layers/579:layers/580:layers/581:layers/582:layers/583:layers/584:layers/585:layers/586:layers/587:layers/588:layers/589:layers/590:layers/591:layers/592:layers/593:layers/594:layers/595:layers/596:layers/597:layers/598:layers/599:layers/600:layers/601:layers/602:layers/603:layers/604:layers/605:layers/606:layers/607:layers/608:layers/609:layers/610:layers/611:layers/612:layers/613:layers/614:layers/615:layers/616:layers/617:layers/618:layers/619:layers/620:layers/621:layers/622:layers/623:layers/624:layers/625:layers/626:layers/627:layers/628:layers/629:layers/630:layers/631:layers/632:layers/633:layers/634:layers/635:layers/636:layers/637:layers/638:layers/639:layers/640:layers/641:layers/642:layers/643:layers/644:layers/645:layers/646:layers/647:layers/648:layers/649:layers/650:layers/651:layers/652:layers/653:layers/654:layers/655:layers/656:layers/657:layers/658:layers/659:layers/660:layers/661:layers/662:layers/663:layers/664:layers/665:layers/666:layers/667:layers/668:layers/669:layers/670:layers/671:layers/672:layers/673:layers/674:layers/675:layers/676:layers/677:layers/678:layers/679:layers/680:layers/681:layers/682:layers/683:layers/684:layers/685:layers/686:layers/687:layers/688:layers/689:layers/690:layers/691:layers/692:layers/693:layers/694:layers/695:layers/696:layers/697:layers/698:layers/699:layers/700:layers/701:layers/702:layers/703:layers/704:layers/705:layers/706:layers/707:layers/708:layers/709:layers/710:layers/711:layers/712:layers/713:layers/714:layers/715:layers/716:layers/717:layers/718:layers/719:layers/720:layers/721:layers/722:layers/723:layers/724:layers/725:layers/726:layers/727:layers/728:layers/729:layers/730:layers/731:layers/732:layers/733:layers/734:layers/735:layers/736:layers/737:layers/738:layers/739:layers/740:layers/741:layers/742:layers/743:layers/744:layers/745:layers/746:layers/747:layers/748:layers/749:layers/750:layers/751:layers/752:layers/753:layers/754:layers/755:layers/756:layers/757:layers/758:layers/759:layers/760:layers/761:layers/762:layers/763:layers/764:layers/765:layers/766:layers/767:layers/768:layers/769:layers/770:layers/771:layers/772:layers/773:layers/774:layers/775:layers/776:layers/777:layers/778:layers/779:layers/780:layers/781:layers/782:layers/783:layers/784:layers/785:layers/786:layers/787:layers/788:layers/789:layers/790:layers/791:layers/792:layers/793:layers/794:layers/795:layers/796:layers/797:layers/798:layers/799:layers/800:layers/801:layers/802:layers/803:layers/804:layers/805:layers/806:layers/807:layers/808:layers/809:layers/810:layers/811:layers/812:layers/813:layers/814:layers/815:layers/816:layers/817:layers/818:layers/819:layers/820:layers/821:layers/822:layers/823:layers/824:layers/825:layers/826:layers/827:layers/828:layers/829:layers/830:layers/831:layers/832:layers/833:layers/834:layers/835:layers/836:layers/837:layers/838:layers/839:layers/840:layers/841:layers/842:layers/843:layers/844:layers/845:layers/846:layers/847:layers/848:layers/849:layers/850:layers/851:layers/852:layers/853:layers/854:layers/855:layers/856:layers/857:layers/858:layers/859:layers/860:layers/861:layers/862:layers/863:layers/864:layers/865:layers/866:layers/867:layers/868:layers/869:layers/870:layers/871:layers/872:layers/873:layers/874:layers/875:layers/876:layers/877:layers/878:layers/879:layers/880:layers/881:layers/882:layers/883:layers/884:layers/885:layers/886:layers/887:layers/888:layers/889:layers/890:layers/891:layers/892:layers/893:layers/894:layers/895:layers/896:layers/897:layers/898:layers/899:layers/900:layers/901:layers/902:layers/903:layers/904:layers/905:layers/906:layers/907:layers/908:layers/909:layers/910:layers/911:layers/912:layers/913:layers/914:layers/915:layers/916:layers/917:layers/918:layers/919:layers/920:layers/921:layers/922:layers/923:layers/924:layers/925:layers/926:layers/927:layers/928:layers/929:layers/930:layers/931:layers/932:layers/933:layers/934:layers/935:layers/936:layers/937:layers/938:layers/939:layers/940:layers/941:layers/942:layers/943:layers/944:layers/945:layers/946:layers/947:layers/948:layers/949:layers/950:layers/951:layers/952:layers/953:layers/954:layers/955:layers/956:layers/957:layers/958:layers/959:layers/960:layers/961:layers/962:layers/963:layers/964:layers/965:layers/966:layers/967:layers/968:layers/969:layers/970:layers/971:layers/972:layers/973:layers/974:layers/975:layers/976:layers/977:layers/978:layers/979:layers/980:layers/981:layers/982:layers/983:layers/984:layers/985:layers/986:layers/987:layers/988:layers/989:layers/990:layers/991:layers/992:layers/993:layers/994:layers/995:layers/996:layers/997:layers/998:layers/999:layers/1000,upperdir=upper,workdir=work") = -1 ENOENT (No such file or directory)
dmesg には次のようなエラーが出ていた。
# dmesg | tail
...
[73448.037990] overlayfs: failed to resolve 'lay': -2
カーネルコードはおおむね次のようになっており、1ページ分(4k)の領域しか確保していないことがわかる(apt で src 取ってきてないので微妙に違うかもしれないけどまあ大まかには一緒でしょうたぶん)。 fs/namespace.c#copy_mount_options
void *copy_mount_options(const void __user * data)
{
int i;
unsigned long size;
char *copy;
if (!data)
return NULL;
copy = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!copy)
return ERR_PTR(-ENOMEM);
/* We only care that *some* data at the address the user
* gave us is valid. Just in case, we'll zero
* the remainder of the page.
*/
/* copy_from_user cannot cross TASK_SIZE ! */
size = TASK_SIZE - (unsigned long)untagged_addr(data);
if (size > PAGE_SIZE)
size = PAGE_SIZE;
i = size - exact_copy_from_user(copy, data, size);
if (!i) {
kfree(copy);
return ERR_PTR(-EFAULT);
}
if (i != PAGE_SIZE)
memset(copy + i, 0, PAGE_SIZE - i);
return copy;
}
まとめ
ということで、docker ビルド時のレイヤー上限は 125。 ただし docker のバージョンによって変わってきた数字(昔は aufs が制限厳しかったので 42 だったらしい)だし、今後も実装依存で変わる可能性もあるので、ドキュメント化はされないと思われる。