libpcut: Occasional build error when doing an incremental build

[Jakub Jermář]

I do occasionally get the following build error when doing an incremental build of a previously build tree:

/usr/local/cross/arm32/bin/arm-linux-gnueabi-ld -n  -T ../../lib/c/arch/arm32/_link.ld -M -Map test-libc.map -o test-libc test/main.test.o test/sprintf.test.o ../../lib/pcut/libpcut.a libc.a  ../../lib/urcu/liburcu.a ../../lib/c/libc.a ../../lib/softint/libsoftint.a ../../lib/softfloat/libsoftfloat.a
test/sprintf.test.o: In function `test_int_as_hex':
/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:88: undefined reference to `pcut_failed_assertion_fmt'
test/sprintf.test.o: In function `test_long_negative_various_padding':
/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:83: undefined reference to `pcut_failed_assertion_fmt'
test/sprintf.test.o: In function `test_int_negative_various_padding':
/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:79: undefined reference to `pcut_failed_assertion_fmt'
test/sprintf.test.o: In function `test_int_various_padding':
/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:75: undefined reference to `pcut_failed_assertion_fmt'
test/sprintf.test.o: In function `test_char_basic':
/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:71: undefined reference to `pcut_failed_assertion_fmt'
test/sprintf.test.o:/home/jermar/software/HelenOS.mainline/uspace/lib/c/test/sprintf.c:67: more undefined references to `pcut_failed_assertion_fmt' follow
../../lib/pcut/libpcut.a(main.o): In function `pcut_main':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:175: undefined reference to `pcut_fix_list_get_real_head'
../../lib/pcut/libpcut.a(main.o): In function `set_setup_teardown_callbacks':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:145: undefined reference to `pcut_get_real_next'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:145: undefined reference to `pcut_get_real_next'
../../lib/pcut/libpcut.a(main.o): In function `pcut_find_by_id':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:77: undefined reference to `pcut_get_real'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:82: undefined reference to `pcut_get_real_next'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:77: undefined reference to `pcut_get_real'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:82: undefined reference to `pcut_get_real_next'
../../lib/pcut/libpcut.a(main.o): In function `pcut_main':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:280: undefined reference to `pcut_get_real_next'
../../lib/pcut/libpcut.a(main.o): In function `run_suite':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:97: undefined reference to `pcut_get_real_next'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:102: undefined reference to `pcut_get_real_next'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:97: undefined reference to `pcut_get_real_next'
../../lib/pcut/libpcut.a(main.o):/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/main.c:102: more undefined references to `pcut_get_real_next' follow
../../lib/pcut/libpcut.a(print.o): In function `pcut_print_tests':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/print.c:75: undefined reference to `pcut_get_real'
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/print.c:75: undefined reference to `pcut_get_real_next'
../../lib/pcut/libpcut.a(tap.o): In function `tap_init':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/report/tap.c:55: undefined reference to `pcut_count_tests'
../../lib/pcut/libpcut.a(xml.o): In function `xml_init':
/home/jermar/software/HelenOS.mainline/uspace/lib/pcut/src/report/xml.c:55: undefined reference to `pcut_count_tests'
../../Makefile.common:368: recipe for target 'test-libc' failed
make[2]: *** [test-libc] Error 1
make[2]: Leaving directory '/home/jermar/software/HelenOS.mainline/uspace/lib/c'
Makefile:259: recipe for target 'lib/c.build-test' failed
make[1]: *** [lib/c.build-test] Error 2
make[1]: Leaving directory '/home/jermar/software/HelenOS.mainline/uspace'
Makefile:50: recipe for target 'all' failed
make: *** [all] Error 2

[Jan Mareš]

I think it's related with rebuilding something that one of the tests is dependent upon. What fixes the problem for me is:

rm uspace/lib/pcut/libpcut.a

then the incremental build finishes alright.

[Vojtech Horky]

This was more complicated than I expected. It seems that the reason for the fault was that libpcut.a was built from *.o files in lib/c due to dependency check and (faulty?) stem expansion.

But I found out that parallel builds are completely broken. By completely I mean that when running following command on an 8-core machine with plenty of RAM, about 25% of full rebuilds terminated with a failure (and not due to insufficient RAM or disk space).

make PROFILE=ia32 HANDS_OFF=y -j

I suspect that the reason is in low resolution of the file timestamps and thus sometimes make rebuilds things that are not necessary, changing files that are already being read by another process.

The attached patch tries to fix these issues but I still encounter some failures with unlimited parallelism. (I have run builds of ia32 with this patch for about 20h randomly touching files and rebuilding with 7-31 parallelism and it looks okay.)

@jermar, could you, please, try this patch if it fixes the problem? Thanks.

[Vojtech Horky]

Fix partial and parallel builds (attempt #1)

[Jiri Svoboda]

I get this error every time I bump the modification time of any *.c file in libc. The above patch seems to work for me.

[Jan Mareš]

The same here. Modification in libc causes the problem. The patch fixed the issue for me too.

[Jakub Jermář]

The patch seems to work for me, even though I don't understand why it removes eg. the dependency of libnic on libdrv and the dependency of libext4 on libblock.

[Vojtech Horky]

Replying to jermar:

why it removes eg. the dependency of libnic on libdrv and the dependency of libext4 on libblock.

Because it is rather a logical dependency than a “rebuild” one. libext4 depends on libblock: when libblock's header changes, we rebuild libext4. This is covered by the Makefile.depend generated by makedepend. But when implementation of libblock changes, i.e. libblock.a is newer than libext4, we do not need to rebuild libext4. We need to rebuild applications using this library and that is covered by their own Makefiles.

Together with the stem matching of %.a: $(OBJECTS) this caused weird errors on parallel incremental builds because the two depending libraries were built from the same object files. At least I think this was the reason (sorry, I completely forgot about this patch and I do not remember all the details now).

Btw, incremental builds with unlimited parallelism are still broken. I think that either (a) make is broken for recursive and parallel builds, (b) we still have some errors in our Makefiles for recursive builds or © with parallel builds we are hitting the resolution of timestamps used by make forcing it to rebuild something twice (thus breaking atomicity for dependent tasks).

[Jakub Jermář]

Fixed in mainline,2506.