GithubHelp home page GithubHelp logo

priv-kweihmann / meta-buildutils Goto Github PK

View Code? Open in Web Editor NEW
10.0 2.0 5.0 125 KB

A collection of build utils to be used with yocto

License: BSD 2-Clause "Simplified" License

BitBake 77.33% Python 22.67%
yocto yocto-layer build-tool build-tools security-hardening poky

meta-buildutils's Issues

swinventory-image: WARNING: core-image-minimal-1.0-r0 do_rootfs: No swinventory for libz1 found 

INHERIT += "swinventory swinventory-image"

among other things:

WARNING: core-image-minimal-1.0-r0 do_rootfs: No swinventory for libz1 found

ls zlib*
zlib-dbg.json  zlib-dev.json  zlib-doc.json  zlib.json  zlib-locale.json  zlib-native.json  zlib-ptest.json  zlib-src.json  zlib-staticdev.json

here are the built packages:

find | grep libz
./armv7at2hf-vfp/libz-dev_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz-ptest_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz-staticdev_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz-doc_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz1_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz-dbg_1.2.11-r0.1_armv7at2hf-vfp.ipk
./armv7at2hf-vfp/libz-src_1.2.11-r0.1_armv7at2hf-vfp.ipk

buildhistory detects, that it's installed in the image.

I guess it has to do with recipe and package name being different.

./meta/recipes-core/zlib/zlib_1.2.11.bb

license-report: also check for HOSTTOOLS_NONFATAL

meta-buildutils/classes/license-report.bbclass

Line 30 in 9ce1ddd

if tool not in d.getVar("HOSTTOOLS").split(" "):
does only check for HOSTTOOLS, but as one can configure the LICENSE_REPORT_FORMATS and there is some double checking it would be enough to have some of the tools in HOSTTOOLS_NONFATAL.

Otherwise setting LICENSE_REPORT_FORMATS = "" would still require setting some HOSTTOOLS and therefore having them installed on the system.
Putting them in HOSTTOOLS_NONFATAL would be more than enough

python-manifest contain non-json chars

the file python3-manifest.json contains multiple lines starting with "#" since warrior release.
Therefore classes/python-package-ident-core.bbclass doesn't work correctly

the JSON object must be str, bytes or bytearray, not 'int'

Whenever I inherit python3-package-ident in my python3 recipes I get an error like this:

ERROR: python3-mastermind-1.0.0+gitAUTOINC+6bea285818-r0 do_package: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_ident_python_packages(d)
     0003:
File: '/home/student/projects/resy-playground/sources/poky/../meta-buildutils/classes/python-package-ident-core.bbclass', lineno: 69, function: do_ident_python_packages
     0065:    import os
     0066:    import bb
     0067:    import dis
     0068:
 *** 0069:    _modules = get_python_modules(d)
     0070:    _package_dir = d.getVar("PKGDEST")
     0071:
     0072:    for _dir in os.listdir(_package_dir):
     0073:        _dir_wo_pn = _dir.replace(d.getVar("PN"), "", 1)
File: '/home/student/projects/resy-playground/sources/poky/../meta-buildutils/classes/python-package-ident-core.bbclass', lineno: 40, function: get_python_modules
     0036:            tmp = i.read()
     0037:            if tmp.startswith("#"):
     0038:                ## Probe for manifest with or without preambel
     0039:                tmp = tmp.find('# EOC') + 6 # EOC + \n -> taken from meta/recipes-devtools/python/python3/create_manifest3.py
 *** 0040:            _module_dict = json.loads(tmp)
     0041:        _strip_path = os.path.join(d.getVar("STAGING_LIBDIR"), d.getVar("PYTHON_DIR"))
     0042:
     0043:        for k,v in _module_dict.items():
     0044:            modules[k] = []
File: '/usr/lib/python3.6/json/__init__.py', lineno: 348, function: loads
     0344:                                  s, 0)
     0345:    else:
     0346:        if not isinstance(s, (bytes, bytearray)):
     0347:            raise TypeError('the JSON object must be str, bytes or bytearray, '
 *** 0348:                            'not {!r}'.format(s.__class__.__name__))
     0349:        s = s.decode(detect_encoding(s), 'surrogatepass')
     0350:
     0351:    if (cls is None and object_hook is None and
     0352:            parse_int is None and parse_float is None and
Exception: TypeError: the JSON object must be str, bytes or bytearray, not 'int'

ERROR: python3-mastermind-1.0.0+gitAUTOINC+6bea285818-r0 do_package: the JSON object must be str, bytes or bytearray, not 'int'
ERROR: python3-mastermind-1.0.0+gitAUTOINC+6bea285818-r0 do_package: Function failed: do_ident_python_packages
ERROR: Logfile of failure stored in: /home/student/projects/resy-playground/build/container-x86-64/tmp/work/core2-64-resy-linux-musl/python3-mastermind/1.0.0+gitAUTOINC+6bea285818-r0/temp/log.do_package.29456
ERROR: Task (/home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb:do_package) failed with exit code '1'
NOTE: Tasks Summary: Attempted 822 tasks of which 812 didn't need to be rerun and 1 failed.
NOTE: The errors for this build are stored in /home/student/projects/resy-playground/build/container-x86-64/tmp/log/error-report/error_report_20190715192845.txt
You can send the errors to a reports server by running:
  send-error-report /home/student/projects/resy-playground/build/container-x86-64/tmp/log/error-report/error_report_20190715192845.txt [-s server]
NOTE: The contents of these logs will be posted in public if you use the above command with the default server. Please ensure you remove any identifying or proprietary information when prompted before sending.

Summary: 1 task failed:
  /home/student/projects/resy-playground/sources/poky/../meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb:do_package
Summary: There were 3 ERROR messages shown, returning a non-zero exit code.

Here[1] you can find samples of my recipes.

[1] https://gitlab.com/meta-layers/meta-desire/tree/master/recipes-desire/python-iot

Exception: ImportError: No module named 'kconfiglib'

Try using inherit kconfig-sanity on linux-yocto recipe, but I getting this error on zeus:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_kconfig_sanity_result(d)
0003:
File: 'meta-buildutils/classes/kconfig-sanity.bbclass', lineno: 191, function: do_kconfig_sanity_result
0187:            if not exp_val:
0188:                exp_val = None
0189:            if fv != exp_val:
0190:                if d.getVar("KCONFIG_SANITY_FRAGMENT_KCONFIG_EXPLAIN") == "1":
*** 0191:                    get_kconfig_explanation(d, fk)
0192:                else:
0193:                    call_logging_function(d, "KCONFIG_SANITY_FRAGMENT_NO_MATCH", "{}{} was set to '{}' - config-file {} configured '{}'".format(d.getVar("KCONFIG_SANITY_CONFIG_PRE"), fk, fv,
os.path.basename(known_symbols[fk]["file"]), exp_val))
0194:        elif fv is not None:
0195:            call_logging_function(d, "KCONFIG_SANITY_FRAGMENT_NEW_SET", "{}{} is set to '{}' but not defined by any config".format(d.getVar("KCONFIG_SANITY_CONFIG_PRE"), fk, fv))
File: 'meta-buildutils/classes/kconfig-sanity.bbclass', lineno: 62, function: get_kconfig_explanation
0058:    return result
0059:
0060:def get_kconfig_explanation(d, symbol):
0061:    import os
*** 0062:    from kconfiglib import Kconfig, TRI_TO_STR, expr_str
0063:    result = ""
0064:    path_bef = os.getcwd()
0065:    for item in d.getVar("KCONFIG_SANITY_KCONFIGS").split(" "):
0066:        if not os.path.exists(item):
Exception: ImportError: No module named 'kconfiglib'

The weird part, I am seeing the class has a dependency to kconfiglib-native, but somehow is unable to find the class

No sca run when using INHERIT += "auto-inherit" in local.conf

Describe the bug
I want to run sca only for recipes under my layer meta-mylayer. For that I followed the documentation here (in the documentation there's a mistake btw, instead of "autoinherit.bbclass", it should be "auto-inherit"). However, when running bitbake, even though I can see the messages "meta-mylayer/recipes-test/test/test.bb: Inherting sca caused by auto-inherit" (there's a typo there, "Inherting -> Inheriting"), there's no sca run for those recipes.
If I explicitly put "inherit sca" into meta-mylayer/recipes-test/test/test.bb, and keep everything else the same, then sca is run for test.bb.

Used layer

  • hash of meta-sca layer - 355398bd7e100da68a313264a73bad8ef41bdf7d
  • hash of meta-buildutils layer - 9fbd6c8

Used configuration

  • This is the SCA related configuration I put in my build/conf/local.conf:
    INHERIT += "auto-inherit"
    AUTO_INHERIT_CONF = "BBClass=sca;props=[auto_inherit_is_at_path(d,'meta-mylayer/',False)]"
    SCA_ENABLE = "1"
    SCA_AUTO_INH_ON_IMAGE = "0"
    SCA_AUTO_INH_ON_RECIPE = "1"
    SCA_AUTO_LICENSE_FILTER = ".*"
    SCA_AVAILABLE_MODULES = "cppcheck"
    SCA_WARNING_LEVEL = "error"
    SCA_GCC_HARDENING = "0"
    SCA_ENABLE_IMAGE_SUMMARY = "0"
    SCA_EXPORT_FINDING_SRC = "1"

swinventory-image: WARNING: core-image-minimal-1.0-r0 do_rootfs: No swinventory for bzip2-replacement-native found

I guess this comes from here:

grep -r bzip2-replacement-native .
./meta/recipes-extended/bzip2/bzip2_1.0.8.bb:PROVIDES_append_class-native = " bzip2-replacement-native"
./meta/recipes-extended/libarchive/libarchive_3.4.3.bb:DEPENDS_BZIP2 = "bzip2-replacement-native"
./meta/recipes-extended/pbzip2/pbzip2_1.1.13.bb:DEPENDS_append_class-native = " bzip2-replacement-native"
./meta/recipes-devtools/elfutils/elfutils_0.182.bb:DEPENDS_BZIP2 = "bzip2-replacement-native"
./meta/recipes-devtools/python/python3_3.9.1.bb:DEPENDS = "bzip2-replacement-native libffi bzip2 openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2"
./meta/recipes-devtools/rpm/rpm_4.16.1.2.bb:DEPENDS_append_class-native = " file-replacement-native bzip2-replacement-native"
./meta/recipes-devtools/dpkg/dpkg.inc:DEPENDS_class-native = "bzip2-replacement-native zlib-native virtual/update-alternatives-native gettext-native perl-native"
./meta/recipes-devtools/cmake/cmake-native_3.19.2.bb:DEPENDS += "bzip2-replacement-native xz-native zlib-native curl-native ncurses-native"
./meta/recipes-devtools/file/file_5.39.bb:DEPENDS_class-native = "bzip2-replacement-native"

It looks like the PROVIDES is ignored.

script: BBMASK generator

reusing a bit of work from the unused script it would be nice to have a script that generates a valid BBMASK string for recipe in layer

  • that provide a never version for a recipe than the layers specified
  • bbappend a recipe from the layer to be protected
  • override classes from the layers specified

swinventory-image: WARNING: core-image-minimal-1.0-r0 do_rootfs: No swinventory for initd-functions found

I guess this comes from here:

grep -r initd-functions .
./meta/conf/distro/include/default-providers.inc:PREFERRED_RPROVIDER_initd-functions ?= "initscripts"
./meta/recipes-core/initscripts/initscripts_1.0.bb:RDEPENDS_${PN} = "initd-functions \
./meta/recipes-core/initscripts/initscripts_1.0.bb:# Recommend pn-functions so that it will be a preferred default provider for initd-functions
./meta/recipes-core/initscripts/initscripts_1.0.bb:RPROVIDES_${PN}-functions = "initd-functions"
./meta/recipes-core/sysvinit/sysvinit_2.98.bb:RDEPENDS_${PN} += "sysvinit-pidof initd-functions base-passwd"
./meta/classes/update-rc.d.bbclass:            d.appendVar('RDEPENDS_' + pkg, ' %sinitd-functions' % (mlprefix))

It looks like RPROVIDES_${PN}-functions = "initd-functions" is ignored

testimage: unwind core dump info

If in a testimage run core dumps occur it would be nice to get further information about these dumps, even without debug info being present in the image.
Add the following

  • oeqa based test that collects core dumps occurring via running the testimage
  • transfer the core dumps to temp directory
  • extract a stack trace for each transferred core dump as a post processing function of testimage and print it to console

script: dot to tree

Add a script that takes the bitbake -g output and generates (using e.g. anytree) a configurable tree.

Following scenarios should be possible

  • specify a task or a recipe name
  • specify a start and an endpoint and generate the tree between those
  • specify a recipe/task and print all recipe/tasks that are require for it
  • specify a recipe/task and print all recipe/tasks that depend on it

inherit python3-package-ident inaccurate?

Here dev-branch is the actual python code: (Note it might be changing quite a bit)

Here is the recipe.

Here is a dependency graph calculated on the board.

and that's the output of the tool

####
WARNING: python3-tcpclientserver-1.0.0+gitAUTOINC+faf7904667-r0 do_package: python3-tcpclientserver uses code from python-packages:python3-io. Please add them to RDEPENDS_${PN}
WARNING: python3-tcpclientserver-1.0.0+gitAUTOINC+faf7904667-r0 do_package: python3-tcpclientserver don't use code from python-packages:python3-configparser,python3-python3. Please remove them to RDEPENDS_${PN}
####

which means only python-packages:python3-io is being used??? I don't see it anywhere in my graph.

just a little typo "please remove them from" not "please remove them to"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.