Comments (31)
ferily7
commented on May 23, 2024
1
Gotcha! I have my environmental variables set up like this in the lambda,
is this the right way to add the environmental variables?
from docker-lambda.
ferily7
commented on May 23, 2024
1
What is the next step to get this resolved then?
from docker-lambda.
vincentsarago
commented on May 23, 2024
1
🤦 no we still have the issue because the official docker image do not match the lambda environment
from docker-lambda.
vincentsarago
commented on May 23, 2024
@ferily7 thanks for your issue.
What version of nodejs are you using?
from docker-lambda.
vincentsarago
commented on May 23, 2024
Note, the latest version in us-west-2 are arn:aws:lambda:us-west-2:524387336408:layer:gdal32-al2:4 or arn:aws:lambda:us-west-2:524387336408:layer:gdal32:4
https://github.com/lambgeo/docker-lambda/blob/master/layer.json#L557
from docker-lambda.
ferily7
commented on May 23, 2024
I'm using Node.js 14.x for the lambda function! I also changed the version to the one you listed, still getting the same error though
from docker-lambda.
vincentsarago
commented on May 23, 2024
for Node.js 14 you have to use arn:aws:lambda:us-west-2:524387336408:layer:gdal32-al2:4
from docker-lambda.
ferily7
commented on May 23, 2024
Actually after I changed the version, I get a different error now Error: ogr2ogr: error while loading shared libraries: liblzma.so.5: cannot open shared object file: No such file or directory
from docker-lambda.
vincentsarago
commented on May 23, 2024
yes I'm seeing this too! I never tried nodejs but seem that there is a difference between the docker image used for the build and the one running the lambda 🤷♂️
The nodejs image from aws seems to have liblzma at the right place tho 🤷♂️
docker run --rm -it lambgeo/lambda-gdal:3.2-al2 bash
bash-4.2# ldd /opt/bin/ogr2ogr | grep "liblzma"
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007ff1d104b000)
docker run --rm -it --entrypoint "" amazon/aws-lambda-nodejs:14 bash
...
Status: Downloaded newer image for amazon/aws-lambda-nodejs:14
bash-4.2# ls -la /lib64/ | grep "liblzma"
lrwxrwxrwx 1 root root 16 Feb 23 11:02 liblzma.so.5 -> liblzma.so.5.2.2
-rwxr-xr-x 1 root root 157400 Jul 26 2018 liblzma.so.5.2.2
from docker-lambda.
ferily7
commented on May 23, 2024
Hmm so does that mean that the library is there then?
from docker-lambda.
vincentsarago
commented on May 23, 2024
Hmm so does that mean that the library is there then?
the library is in the official AWS Lambda NodeJS 14x image .... but maybe not in the lambda runtime which is really weird!
from docker-lambda.
ferily7
commented on May 23, 2024
Strange... Is there any way I could resolve this issue?
from docker-lambda.
vincentsarago
commented on May 23, 2024
if you could list all the files found in /lib64 and /lib we could compare with what we have in our building env.
It's always hard to keep a track of what is shipped in the AWS runtime images 😭
from docker-lambda.
ferily7
commented on May 23, 2024
How would I find the list of all the files in /lib64 and /lib?
from docker-lambda.
vincentsarago
commented on May 23, 2024
create a lambda function that return the content of /lib64!
from docker-lambda.
ferily7
commented on May 23, 2024
I'm not too sure how to do that... Is that the only way to get the files in /lib64 or /lib?
from docker-lambda.
vincentsarago
commented on May 23, 2024
Is that the only way to get the files in /lib64 or /lib?
Here is how I do in it python
import json
from pathlib import Path
def lambda_handler(event, context):
p = Path('/lib64').glob('**/*.so')
files = [str(x) for x in p]
print(files)
return {
'statusCode': 200,
'body': json.dumps(files)
}/lib64/libnss_files-2.26.so
/lib64/libSegFault.so
/lib64/libmemusage.so
/lib64/libfreeblpriv3.so
/lib64/libpcprofile.so
/lib64/libnspr4.so
/lib64/librt-2.26.so
/lib64/libanl-2.26.so
/lib64/p11-kit-trust.so
/lib64/libfreebl3.so
/lib64/libnssckbi.so
/lib64/libBrokenLocale-2.26.so
/lib64/libthread_db-1.0.so
/lib64/libc-2.26.so
/lib64/libcidn-2.26.so
/lib64/libnss_compat-2.26.so
/lib64/libnsl-2.26.so
/lib64/libresolv-2.26.so
/lib64/libnssutil3.so
/lib64/libplc4.so
/lib64/ld-2.26.so
/lib64/p11-kit-proxy.so
/lib64/libplds4.so
/lib64/libnss_dns-2.26.so
/lib64/libutil-2.26.so
/lib64/libdl-2.26.so
/lib64/libm-2.26.so
/lib64/libpthread-2.26.so
/lib64/libmvec-2.26.so
/lib64/gconv/IBM904.so
/lib64/gconv/ANSI_X3.110.so
/lib64/gconv/IBM864.so
/lib64/gconv/IBM4971.so
/lib64/gconv/HP-TURKISH8.so
/lib64/gconv/IBM1157.so
/lib64/gconv/TCVN5712-1.so
/lib64/gconv/ISO_5427.so
/lib64/gconv/IBM905.so
/lib64/gconv/ISO8859-15.so
/lib64/gconv/GBK.so
/lib64/gconv/IBM285.so
/lib64/gconv/ISO-2022-KR.so
/lib64/gconv/MAC-IS.so
/lib64/gconv/IBM423.so
/lib64/gconv/EBCDIC-ES-S.so
/lib64/gconv/IBM290.so
/lib64/gconv/CP1258.so
/lib64/gconv/HP-GREEK8.so
/lib64/gconv/IBM4899.so
/lib64/gconv/ISO-IR-197.so
/lib64/gconv/IBM852.so
/lib64/gconv/IBM1025.so
/lib64/gconv/IBM281.so
/lib64/gconv/ISO8859-6.so
/lib64/gconv/ISIRI-3342.so
/lib64/gconv/ISO8859-10.so
/lib64/gconv/IBM1137.so
/lib64/gconv/IBM1167.so
/lib64/gconv/IBM1122.so
/lib64/gconv/DEC-MCS.so
/lib64/gconv/IBM855.so
/lib64/gconv/GB18030.so
/lib64/gconv/EUC-KR.so
/lib64/gconv/CP1255.so
/lib64/gconv/ISO8859-5.so
/lib64/gconv/EUC-CN.so
/lib64/gconv/EBCDIC-ES-A.so
/lib64/gconv/IBM865.so
/lib64/gconv/IBM868.so
/lib64/gconv/CP770.so
/lib64/gconv/PT154.so
/lib64/gconv/IBM918.so
/lib64/gconv/IBM1153.so
/lib64/gconv/IBM1371.so
/lib64/gconv/EBCDIC-CA-FR.so
/lib64/gconv/IBM1148.so
/lib64/gconv/ISO8859-8.so
/lib64/gconv/CP774.so
/lib64/gconv/INIS-8.so
/lib64/gconv/IBM856.so
/lib64/gconv/SAMI-WS2.so
/lib64/gconv/ISO-2022-JP-3.so
/lib64/gconv/MIK.so
/lib64/gconv/IBM273.so
/lib64/gconv/UNICODE.so
/lib64/gconv/IBM857.so
/lib64/gconv/IBM1123.so
/lib64/gconv/CP773.so
/lib64/gconv/ISO_5428.so
/lib64/gconv/UHC.so
/lib64/gconv/IBM870.so
/lib64/gconv/IBM1004.so
/lib64/gconv/MAC-SAMI.so
/lib64/gconv/IBM16804.so
/lib64/gconv/IBM274.so
/lib64/gconv/SJIS.so
/lib64/gconv/IBM901.so
/lib64/gconv/HP-THAI8.so
/lib64/gconv/TIS-620.so
/lib64/gconv/IBM1155.so
/lib64/gconv/IBM297.so
/lib64/gconv/IBM1149.so
/lib64/gconv/IBM803.so
/lib64/gconv/EUC-JISX0213.so
/lib64/gconv/IBM12712.so
/lib64/gconv/VISCII.so
/lib64/gconv/IBM1146.so
/lib64/gconv/GOST_19768-74.so
/lib64/gconv/UTF-7.so
/lib64/gconv/UTF-32.so
/lib64/gconv/BIG5HKSCS.so
/lib64/gconv/JOHAB.so
/lib64/gconv/IBM277.so
/lib64/gconv/SHIFT_JISX0213.so
/lib64/gconv/ISO8859-4.so
/lib64/gconv/IBM1008.so
/lib64/gconv/CP1253.so
/lib64/gconv/CP1257.so
/lib64/gconv/IBM256.so
/lib64/gconv/KOI-8.so
/lib64/gconv/KOI8-U.so
/lib64/gconv/libJIS.so
/lib64/gconv/CP775.so
/lib64/gconv/T.61.so
/lib64/gconv/IBM9030.so
/lib64/gconv/EBCDIC-IT.so
/lib64/gconv/IBM1141.so
/lib64/gconv/IBM1144.so
/lib64/gconv/IBM875.so
/lib64/gconv/GEORGIAN-ACADEMY.so
/lib64/gconv/ARMSCII-8.so
/lib64/gconv/IBM935.so
/lib64/gconv/IBM1133.so
/lib64/gconv/HP-ROMAN8.so
/lib64/gconv/CP737.so
/lib64/gconv/ISO_6937.so
/lib64/gconv/IBM874.so
/lib64/gconv/ISO8859-9.so
/lib64/gconv/IBM850.so
/lib64/gconv/EBCDIC-FR.so
/lib64/gconv/ISO8859-11.so
/lib64/gconv/GREEK7-OLD.so
/lib64/gconv/IBM1364.so
/lib64/gconv/EBCDIC-FI-SE.so
/lib64/gconv/IBM278.so
/lib64/gconv/IBM275.so
/lib64/gconv/IBM1163.so
/lib64/gconv/GREEK-CCITT.so
/lib64/gconv/CP1250.so
/lib64/gconv/GEORGIAN-PS.so
/lib64/gconv/IBM863.so
/lib64/gconv/ISO8859-13.so
/lib64/gconv/CP772.so
/lib64/gconv/ISO-2022-JP.so
/lib64/gconv/RK1048.so
/lib64/gconv/CP1254.so
/lib64/gconv/ISO_5427-EXT.so
/lib64/gconv/IBM1161.so
/lib64/gconv/IBM1008_420.so
/lib64/gconv/ISO_6937-2.so
/lib64/gconv/CP10007.so
/lib64/gconv/MACINTOSH.so
/lib64/gconv/IBM1046.so
/lib64/gconv/EBCDIC-US.so
/lib64/gconv/KOI8-R.so
/lib64/gconv/CP932.so
/lib64/gconv/MAC-CENTRALEUROPE.so
/lib64/gconv/IBM1156.so
/lib64/gconv/EBCDIC-FI-SE-A.so
/lib64/gconv/KOI8-T.so
/lib64/gconv/IBM038.so
/lib64/gconv/ISO8859-16.so
/lib64/gconv/IBM1132.so
/lib64/gconv/EUC-JP-MS.so
/lib64/gconv/IBM1142.so
/lib64/gconv/CP1251.so
/lib64/gconv/IBM1164.so
/lib64/gconv/IBM1140.so
/lib64/gconv/IBM939.so
/lib64/gconv/libISOIR165.so
/lib64/gconv/IBM1147.so
/lib64/gconv/CSN_369103.so
/lib64/gconv/ISO-2022-CN.so
/lib64/gconv/IBM858.so
/lib64/gconv/BRF.so
/lib64/gconv/IBM921.so
/lib64/gconv/IBM943.so
/lib64/gconv/IBM284.so
/lib64/gconv/CP1252.so
/lib64/gconv/libKSC.so
/lib64/gconv/EBCDIC-AT-DE-A.so
/lib64/gconv/IBM851.so
/lib64/gconv/CP1256.so
/lib64/gconv/IBM902.so
/lib64/gconv/IBM1112.so
/lib64/gconv/IBM930.so
/lib64/gconv/UTF-16.so
/lib64/gconv/HP-ROMAN9.so
/lib64/gconv/IBM1145.so
/lib64/gconv/IBM866.so
/lib64/gconv/GBBIG5.so
/lib64/gconv/IBM4517.so
/lib64/gconv/IBM9066.so
/lib64/gconv/IBM871.so
/lib64/gconv/ISO8859-2.so
/lib64/gconv/ISO-IR-209.so
/lib64/gconv/IBM1399.so
/lib64/gconv/IBM861.so
/lib64/gconv/LATIN-GREEK-1.so
/lib64/gconv/IBM932.so
/lib64/gconv/IBM1124.so
/lib64/gconv/IEC_P27-1.so
/lib64/gconv/libJISX0213.so
/lib64/gconv/ASMO_449.so
/lib64/gconv/IBM5347.so
/lib64/gconv/IBM1160.so
/lib64/gconv/IBM420.so
/lib64/gconv/IBM9448.so
/lib64/gconv/MAC-UK.so
/lib64/gconv/IBM891.so
/lib64/gconv/IBM1154.so
/lib64/gconv/EBCDIC-DK-NO.so
/lib64/gconv/libGB.so
/lib64/gconv/EBCDIC-AT-DE.so
/lib64/gconv/IBM933.so
/lib64/gconv/ISO8859-3.so
/lib64/gconv/ECMA-CYRILLIC.so
/lib64/gconv/INIS.so
/lib64/gconv/ISO8859-1.so
/lib64/gconv/EBCDIC-PT.so
/lib64/gconv/IBM424.so
/lib64/gconv/IBM1130.so
/lib64/gconv/LATIN-GREEK.so
/lib64/gconv/GBGBK.so
/lib64/gconv/EUC-TW.so
/lib64/gconv/IBM903.so
/lib64/gconv/IBM1162.so
/lib64/gconv/INIS-CYRILLIC.so
/lib64/gconv/IBM880.so
/lib64/gconv/libCNS.so
/lib64/gconv/IBM437.so
/lib64/gconv/ISO_11548-1.so
/lib64/gconv/IBM1143.so
/lib64/gconv/IBM500.so
/lib64/gconv/IBM1047.so
/lib64/gconv/IBM862.so
/lib64/gconv/ISO8859-7.so
/lib64/gconv/EUC-JP.so
/lib64/gconv/IBM869.so
/lib64/gconv/ISO-2022-CN-EXT.so
/lib64/gconv/ISO_10367-BOX.so
/lib64/gconv/IBM1129.so
/lib64/gconv/CP1125.so
/lib64/gconv/IBM860.so
/lib64/gconv/IBM280.so
/lib64/gconv/EBCDIC-DK-NO-A.so
/lib64/gconv/ISO8859-9E.so
/lib64/gconv/IBM1390.so
/lib64/gconv/IBM037.so
/lib64/gconv/IBM1166.so
/lib64/gconv/IBM937.so
/lib64/gconv/ISO646.so
/lib64/gconv/EBCDIC-UK.so
/lib64/gconv/CWI.so
/lib64/gconv/GREEK7.so
/lib64/gconv/ISO_2033.so
/lib64/gconv/IBM4909.so
/lib64/gconv/KOI8-RU.so
/lib64/gconv/ISO8859-14.so
/lib64/gconv/IBM922.so
/lib64/gconv/BIG5.so
/lib64/gconv/IBM1097.so
/lib64/gconv/NATS-SEFI.so
/lib64/gconv/IBM1158.so
/lib64/gconv/TSCII.so
/lib64/gconv/CP771.so
/lib64/gconv/IBM866NAV.so
/lib64/gconv/IBM1388.so
/lib64/gconv/IBM1026.so
/lib64/gconv/EBCDIC-ES.so
/lib64/gconv/NATS-DANO.so
/lib64/gconv/EBCDIC-IS-FRISS.so
/lib64/audit/sotruss-lib.so
/lib64/krb5/plugins/tls/k5tls.so
/lib64/openssl/engines/libaep.so
/lib64/openssl/engines/lib4758cca.so
/lib64/openssl/engines/libubsec.so
/lib64/openssl/engines/libpadlock.so
/lib64/openssl/engines/libchil.so
/lib64/openssl/engines/libcapi.so
/lib64/openssl/engines/libatalla.so
/lib64/openssl/engines/libcswift.so
/lib64/openssl/engines/libnuron.so
/lib64/openssl/engines/libgmp.so
/lib64/openssl/engines/libsureware.so
/lib64/security/pam_cap.so
/lib64/pkcs11/p11-kit-trust.so
as you can see in ☝️ there is not liblzma in the lambda environment 🤷♂️
ahhhhh if I run ldd /opt/bin/ogr2ogr in the lambda env (python3.8) here is what I have 👇
linux-vdso.so.1 (0x00007ffe70e9e000)
libgdal.so => /opt/bin/../lib/libgdal.so (0x00007f1e1f62e000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f1e1f1d9000)
libgeos_c.so.1 => /opt/bin/../lib/libgeos_c.so.1 (0x00007f1e1efa6000)
libwebp.so.7 => /opt/bin/../lib/libwebp.so.7 (0x00007f1e1ed3c000)
libexpat.so.1 => /opt/bin/../lib/libexpat.so.1 (0x00007f1e1eb0c000)
libopenjp2.so.7 => /opt/bin/../lib/libopenjp2.so.7 (0x00007f1e1e8b4000)
libnetcdf.so.18 => /opt/bin/../lib/libnetcdf.so.18 (0x00007f1e1e581000)
libhdf5.so.200 => /opt/bin/../lib/libhdf5.so.200 (0x00007f1e1deaf000)
libmfhdf.so.0 => /opt/bin/../lib/libmfhdf.so.0 (0x00007f1e1dc85000)
libdf.so.0 => /opt/bin/../lib/libdf.so.0 (0x00007f1e1d9d4000)
libjpeg.so.62 => /opt/bin/../lib/libjpeg.so.62 (0x00007f1e1d740000)
libgeotiff.so.5 => /opt/bin/../lib/libgeotiff.so.5 (0x00007f1e1d50c000)
libpng16.so.16 => /opt/bin/../lib/libpng16.so.16 (0x00007f1e1d2db000)
libpq.so.5 => /opt/bin/../lib/libpq.so.5 (0x00007f1e1d092000)
libzstd.so.1 => /opt/bin/../lib/libzstd.so.1 (0x00007f1e1ce0a000)
libproj.so.19 => /opt/bin/../lib/libproj.so.19 (0x00007f1e1c946000)
libsqlite3.so.0 => /opt/bin/../lib/libsqlite3.so.0 (0x00007f1e1c635000)
libtiff.so.5 => /opt/bin/../lib/libtiff.so.5 (0x00007f1e1c3b7000)
libdeflate.so.0 => /opt/bin/../lib/libdeflate.so.0 (0x00007f1e1c1aa000)
libz.so.1 => /lib64/libz.so.1 (0x00007f1e1bf95000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1e1bd77000)
librt.so.1 => /lib64/librt.so.1 (0x00007f1e1bb6f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f1e1b96b000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f1e1b707000)
libcurl.so.4 => /opt/bin/../lib/libcurl.so.4 (0x00007f1e1b48e000)
libxml2.so.2 => /opt/bin/../lib/libxml2.so.2 (0x00007f1e1b144000)
liblzma.so.5 => /var/lang/lib/liblzma.so.5 (0x00007f1e1af1e000)
libm.so.6 => /lib64/libm.so.6 (0x00007f1e1abde000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f1e1a85c000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1e1a646000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1e1a29b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1e201e5000)
libgeos-3.9.0.so => /opt/lib/libgeos-3.9.0.so (0x00007f1e19ea4000)
libhdf5_hl.so.200 => /opt/lib/libhdf5_hl.so.200 (0x00007f1e19c82000)
libsz.so.2 => /opt/lib/libsz.so.2 (0x00007f1e19a6e000)
libnghttp2.so.14 => /opt/lib/libnghttp2.so.14 (0x00007f1e19845000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007f1e195d6000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f1e1938a000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f1e190a6000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f1e18ea2000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f1e18c71000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f1e18a62000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f1e1885e000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f1e18648000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f1e18421000)
liblzma.so.5 => /var/lang/lib/liblzma.so.5 (0x00007f1e1af1e000)
it's interesting that this is not in /lib64. I guess nodejs env don't have /var/lang/lib/liblzma.so.5.
To Do
ldd /opt/bin/ogr2ogrin the NodeJS lambda env
from docker-lambda.
vincentsarago
commented on May 23, 2024
Alright I did it
const { exec } = require("child_process");
exports.handler = (event) => {
exec("ldd /opt/bin/ogr2ogr", (error, stdout, stderr) => {
if (error) {
console.log(`error: ${error.message}`);
}
if (stderr) {
console.log(`stderr: ${stderr}`);
}
console.log(`stdout: ${stdout}`);
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(stdout),
};
return response;
});
}; libgdal.so => /opt/bin/../lib/libgdal.so (0x00007f9c72879000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f9c72424000)
libgeos_c.so.1 => /opt/bin/../lib/libgeos_c.so.1 (0x00007f9c721f1000)
libwebp.so.7 => /opt/bin/../lib/libwebp.so.7 (0x00007f9c71f87000)
libexpat.so.1 => /opt/bin/../lib/libexpat.so.1 (0x00007f9c71d57000)
libopenjp2.so.7 => /opt/bin/../lib/libopenjp2.so.7 (0x00007f9c71aff000)
libnetcdf.so.18 => /opt/bin/../lib/libnetcdf.so.18 (0x00007f9c717cc000)
libhdf5.so.200 => /opt/bin/../lib/libhdf5.so.200 (0x00007f9c710fa000)
libmfhdf.so.0 => /opt/bin/../lib/libmfhdf.so.0 (0x00007f9c70ed0000)
libdf.so.0 => /opt/bin/../lib/libdf.so.0 (0x00007f9c70c1f000)
libjpeg.so.62 => /opt/bin/../lib/libjpeg.so.62 (0x00007f9c7098b000)
libgeotiff.so.5 => /opt/bin/../lib/libgeotiff.so.5 (0x00007f9c70757000)
libpng16.so.16 => /opt/bin/../lib/libpng16.so.16 (0x00007f9c70526000)
libpq.so.5 => /opt/bin/../lib/libpq.so.5 (0x00007f9c702dd000)
libzstd.so.1 => /opt/bin/../lib/libzstd.so.1 (0x00007f9c70055000)
libproj.so.19 => /opt/bin/../lib/libproj.so.19 (0x00007f9c6fb91000)
libsqlite3.so.0 => /opt/bin/../lib/libsqlite3.so.0 (0x00007f9c6f880000)
libtiff.so.5 => /opt/bin/../lib/libtiff.so.5 (0x00007f9c6f602000)
libdeflate.so.0 => /opt/bin/../lib/libdeflate.so.0 (0x00007f9c6f3f5000)
libz.so.1 => /lib64/libz.so.1 (0x00007f9c6f1e0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9c6efc2000)
librt.so.1 => /lib64/librt.so.1 (0x00007f9c6edba000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f9c6ebb6000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f9c6e952000)
libcurl.so.4 => /opt/bin/../lib/libcurl.so.4 (0x00007f9c6e6d9000)
libxml2.so.2 => /opt/bin/../lib/libxml2.so.2 (0x00007f9c6e38f000)
liblzma.so.5 => not found
libm.so.6 => /lib64/libm.so.6 (0x00007f9c6e04f000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f9c6dccd000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f9c6dab7000)
libc.so.6 => /lib64/libc.so.6 (0x00007f9c6d70c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9c73430000)
liblzma.so.5 => not found
libgeos-3.9.0.so => /opt/lib/libgeos-3.9.0.so (0x00007f9c6d315000)
libhdf5_hl.so.200 => /opt/lib/libhdf5_hl.so.200 (0x00007f9c6d0f3000)
libsz.so.2 => /opt/lib/libsz.so.2 (0x00007f9c6cedf000)
libnghttp2.so.14 => /opt/lib/libnghttp2.so.14 (0x00007f9c6ccb6000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007f9c6ca47000)
liblzma.so.5 => not found
liblzma.so.5 => not found
liblzma.so.5 => not found
liblzma.so.5 => not found
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f9c6c7fb000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f9c6c517000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f9c6c313000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f9c6c0e2000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f9c6bed3000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f9c6bccf000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f9c6bab9000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f9c6b892000)
Only liblzma is missing 🤷♂️ I think I could add it manually in the built but I just hope that other env don't have other missing libs
from docker-lambda.
ferily7
commented on May 23, 2024
Ah I see so it's missing from the lib then? I'm just wondering if that one is added, will there be any more missing libraries that I will get an error for
from docker-lambda.
vincentsarago
commented on May 23, 2024
Ah I see so it's missing from the lib then?
yeah for an unknown reason, liblzma is present in AWS official build images, in python lambda env ... but not in NodeJS lambda env 🤷♂️
It seems to be the only one missing
from docker-lambda.
vincentsarago
commented on May 23, 2024
I'm still not 💯 how this has to be resolved. we could compile liblzma ourself but it's really weird that it is missing in the nodejs14 lambda runtime image. Might be worth to open a ticket on AWS
from docker-lambda.
danielsharvey
commented on May 23, 2024
I will see about submitting a ticket to AWS (not sure where for these types of issues).
In the meantime, my app ships a layer that just contains the two missing files:
/lib64/liblzma.so.5.2.2
/lib64/liblzma.so.5
extracted from the official NodeJS Lambda image amazon/aws-lambda-nodejs:14.
from docker-lambda.
vincentsarago
commented on May 23, 2024
fixed in latest release of AWS Lambda nodejs16
from docker-lambda.
JackTiber
commented on May 23, 2024
Any update on this issue? I have been trying to add liblzma or disable it the gdal build command to even just use gdalinfo from the node process, but nothing seems to work.
from docker-lambda.
vincentsarago
commented on May 23, 2024
@JackTiber I didn't had time yet to work on this.
you could try doing #37 (comment)
The issue is that the build image has liblzma but the lambda end doesn't for node js. We need to put liblzma.so in the package/layer to fix this.
from docker-lambda.
JackTiber
commented on May 23, 2024
Thanks @vincentsarago for getting back to me! Yup I actually did that yesterday after finding this thread and it worked with the lambda for nodejs:16. @danielsharvey if you were able to submit a ticket, let me know and I will drop one in as well. Spent entirely too long trying to nail down the missing library issue because it is documented as being there!
from docker-lambda.
arobdog
commented on May 23, 2024
Hi @JackTiber @danielsharvey @vincentsarago , I have also run into this issue. I am a little stuck with trying to implement the solution metioned in #37 (comment). I am having trouble figuring out how to extract the files from the official aws lambda image for amazon/aws-lambda-nodejs:14 so that I can create a separate lambda layer with just those files in it. I have not used docker too much so any pointers or advice would be greatly appreciated.
from docker-lambda.
danielsharvey
commented on May 23, 2024
Sorry, missed the earlier comments. I have CDK code which performs this - I will extract and post here.
from docker-lambda.
JackTiber
commented on May 23, 2024
@arobdog I setup a Lambda layer using a Dockerfile to extract the needed lib files and put them in the expected location. Dockerfile is below for reference.
FROM public.ecr.aws/lambda/nodejs:latest-x86_64
# Work Directory
WORKDIR /asset
# Create output directory
RUN mkdir -p /asset/lib
# Move liblzma librarie files to the layer output
RUN cp /lib64/liblzma.so.5 /asset/lib/
RUN cp /lib64/liblzma.so.5.2.5 /asset/lib/
Then I just add this as another layer to the expected runtime in addition to the OSGeo layer and it works. Hope this helps!
from docker-lambda.
arobdog
commented on May 23, 2024
@JackTiber I managed to get it working. Thanks for your help :)
from docker-lambda.
tfrascari77
commented on May 23, 2024
I am running into the same issue and even building a layer with liblzma.so.5 and liblzma.so.5.2.2 in it does't solve the issue for me.
I confirmed that both libraries are in opt/lib after adding the layer.
2024-03-10T20:48:09.788Z 2b781264-d317-4b02-9127-333b92d80bb3 INFO ls -l /opt/lib
total 48603
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libblosc.so -> libblosc.so.1
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libblosc.so.1 -> libblosc.so.1.21.0
-rwxr-xr-x 1 root root 161960 Feb 2 16:45 libblosc.so.1.21.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libcurl.so -> libcurl.so.4.7.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libcurl.so.4 -> libcurl.so.4.7.0
-rwxr-xr-x 1 root root 495600 Feb 2 16:45 libcurl.so.4.7.0
lrwxrwxrwx 1 root root 15 Mar 10 20:44 libdeflate.so -> libdeflate.so.0
-rwxr-xr-x 1 root root 68960 Feb 2 16:45 libdeflate.so.0
lrwxrwxrwx 1 root root 14 Mar 10 20:44 libdf.so -> libdf.so.0.0.0
lrwxrwxrwx 1 root root 14 Mar 10 20:44 libdf.so.0 -> libdf.so.0.0.0
-rwxr-xr-x 1 root root 555912 Feb 2 16:45 libdf.so.0.0.0
lrwxrwxrwx 1 root root 22 Mar 10 20:44 libecpg_compat.so -> libecpg_compat.so.3.15
lrwxrwxrwx 1 root root 22 Mar 10 20:44 libecpg_compat.so.3 -> libecpg_compat.so.3.15
-rwxr-xr-x 1 root root 31384 Feb 2 16:45 libecpg_compat.so.3.15
lrwxrwxrwx 1 root root 15 Mar 10 20:44 libecpg.so -> libecpg.so.6.15
lrwxrwxrwx 1 root root 15 Mar 10 20:44 libecpg.so.6 -> libecpg.so.6.15
-rwxr-xr-x 1 root root 93184 Feb 2 16:45 libecpg.so.6.15
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libexpat.so -> libexpat.so.1.6.12
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libexpat.so.1 -> libexpat.so.1.6.12
-rwxr-xr-x 1 root root 201536 Feb 2 16:45 libexpat.so.1.6.12
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libgdal.so -> libgdal.so.34
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libgdal.so.34 -> libgdal.so.34.3.8.3
-rwxr-xr-x 1 root root 26142608 Feb 2 16:45 libgdal.so.34.3.8.3
lrwxrwxrwx 1 root root 14 Mar 10 20:44 libgeos_c.so -> libgeos_c.so.1
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libgeos_c.so.1 -> libgeos_c.so.1.18.1
-rwxr-xr-x 1 root root 340280 Feb 2 16:45 libgeos_c.so.1.18.1
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libgeos.so -> libgeos.so.3.12.1
-rwxr-xr-x 1 root root 3721800 Feb 2 16:45 libgeos.so.3.12.1
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libgeotiff.so -> libgeotiff.so.5.2.0
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libgeotiff.so.5 -> libgeotiff.so.5.2.0
-rwxr-xr-x 1 root root 211968 Feb 2 16:45 libgeotiff.so.5.2.0
-rwxr-xr-x 1 root root 66840 Feb 2 16:45 libh5bzip2.so
lrwxrwxrwx 1 root root 22 Mar 10 20:44 libhdf5_cpp.so -> libhdf5_cpp.so.200.0.0
lrwxrwxrwx 1 root root 22 Mar 10 20:44 libhdf5_cpp.so.200 -> libhdf5_cpp.so.200.0.0
-rwxr-xr-x 1 root root 441160 Feb 2 16:45 libhdf5_cpp.so.200.0.0
lrwxrwxrwx 1 root root 25 Mar 10 20:44 libhdf5_hl_cpp.so -> libhdf5_hl_cpp.so.200.0.0
lrwxrwxrwx 1 root root 25 Mar 10 20:44 libhdf5_hl_cpp.so.200 -> libhdf5_hl_cpp.so.200.0.0
-rwxr-xr-x 1 root root 14536 Feb 2 16:45 libhdf5_hl_cpp.so.200.0.0
lrwxrwxrwx 1 root root 21 Mar 10 20:44 libhdf5_hl.so -> libhdf5_hl.so.200.0.0
lrwxrwxrwx 1 root root 21 Mar 10 20:44 libhdf5_hl.so.200 -> libhdf5_hl.so.200.0.0
-rwxr-xr-x 1 root root 134720 Feb 2 16:45 libhdf5_hl.so.200.0.0
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libhdf5.so -> libhdf5.so.200.0.0
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libhdf5.so.200 -> libhdf5.so.200.0.0
-rwxr-xr-x 1 root root 5040456 Feb 2 16:45 libhdf5.so.200.0.0
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libjpeg.so -> libjpeg.so.62
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libjpeg.so.62 -> libjpeg.so.62.3.0
-rwxr-xr-x 1 root root 682624 Feb 2 16:45 libjpeg.so.62.3.0
-rwxr-xr-x 1 root root 431648 Feb 2 16:45 libLercLib.so
-rwxr-xr-x 1 root root 202400 Mar 10 20:28 liblzma.so.5
-rwxr-xr-x 1 root root 202400 Mar 10 20:28 liblzma.so.5.2.2
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libmfhdf.so -> libmfhdf.so.0.0.0
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libmfhdf.so.0 -> libmfhdf.so.0.0.0
-rwxr-xr-x 1 root root 167912 Feb 2 16:45 libmfhdf.so.0.0.0
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libnetcdf.so -> libnetcdf.so.18.0.0
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libnetcdf.so.18 -> libnetcdf.so.18.0.0
-rwxr-xr-x 1 root root 1249984 Feb 2 16:45 libnetcdf.so.18.0.0
lrwxrwxrwx 1 root root 21 Mar 10 20:44 libnghttp2.so -> libnghttp2.so.14.20.1
lrwxrwxrwx 1 root root 21 Mar 10 20:44 libnghttp2.so.14 -> libnghttp2.so.14.20.1
-rwxr-xr-x 1 root root 177336 Feb 2 16:45 libnghttp2.so.14.20.1
lrwxrwxrwx 1 root root 15 Mar 10 20:44 libopenjp2.so -> libopenjp2.so.7
-rwxr-xr-x 1 root root 379680 Feb 2 16:45 libopenjp2.so.2.5.0
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libopenjp2.so.7 -> libopenjp2.so.2.5.0
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libpgtypes.so -> libpgtypes.so.3.15
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libpgtypes.so.3 -> libpgtypes.so.3.15
-rwxr-xr-x 1 root root 76416 Feb 2 16:45 libpgtypes.so.3.15
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libpng16.so -> libpng16.so.16.37.0
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libpng16.so.16 -> libpng16.so.16.37.0
-rwxr-xr-x 1 root root 210656 Feb 2 16:45 libpng16.so.16.37.0
lrwxrwxrwx 1 root root 11 Mar 10 20:44 libpng.so -> libpng16.so
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libpq.so -> libpq.so.5.15
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libpq.so.5 -> libpq.so.5.15
-rwxr-xr-x 1 root root 311112 Feb 2 16:45 libpq.so.5.15
lrwxrwxrwx 1 root root 13 Mar 10 20:44 libproj.so -> libproj.so.25
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libproj.so.25 -> libproj.so.25.9.3.1
-rwxr-xr-x 1 root root 3824872 Feb 2 16:45 libproj.so.25.9.3.1
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libsqlite3.so -> libsqlite3.so.0.8.6
lrwxrwxrwx 1 root root 19 Mar 10 20:44 libsqlite3.so.0 -> libsqlite3.so.0.8.6
-rwxr-xr-x 1 root root 1125256 Feb 2 16:45 libsqlite3.so.0.8.6
lrwxrwxrwx 1 root root 14 Mar 10 20:44 libsz.so -> libsz.so.2.0.0
lrwxrwxrwx 1 root root 14 Mar 10 20:44 libsz.so.2 -> libsz.so.2.0.0
-rwxr-xr-x 1 root root 52320 Feb 2 16:45 libsz.so.2.0.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libtiff.so -> libtiff.so.6.0.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libtiff.so.6 -> libtiff.so.6.0.0
-rwxr-xr-x 1 root root 536288 Feb 2 16:45 libtiff.so.6.0.0
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libtiffxx.so -> libtiffxx.so.6.0.0
lrwxrwxrwx 1 root root 18 Mar 10 20:44 libtiffxx.so.6 -> libtiffxx.so.6.0.0
-rwxr-xr-x 1 root root 10312 Feb 2 16:45 libtiffxx.so.6.0.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libwebp.so -> libwebp.so.7.1.0
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libwebp.so.7 -> libwebp.so.7.1.0
-rwxr-xr-x 1 root root 431200 Feb 2 16:45 libwebp.so.7.1.0
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libxml2.so -> libxml2.so.2.9.10
lrwxrwxrwx 1 root root 17 Mar 10 20:44 libxml2.so.2 -> libxml2.so.2.9.10
-rwxr-xr-x 1 root root 1399088 Feb 2 16:45 libxml2.so.2.9.10
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libzstd.so -> libzstd.so.1.4.5
lrwxrwxrwx 1 root root 16 Mar 10 20:44 libzstd.so.1 -> libzstd.so.1.4.5
-rwxr-xr-x 1 root root 565704 Feb 2 16:45 libzstd.so.1.4.5
But the ldd command still can't find them...
2024-03-10T20:48:09.789Z 2b781264-d317-4b02-9127-333b92d80bb3 INFO ldd /opt/bin/ogr2ogr:
linux-vdso.so.1 (0x00007ffe555d9000)
libgdal.so.34 => /opt/bin/../lib/libgdal.so.34 (0x00007fe30aa33000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fe30a6b1000)
libm.so.6 => /lib64/libm.so.6 (0x00007fe30a371000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe30a15b000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe309dae000)
libz.so.1 => /lib64/libz.so.1 (0x00007fe309b99000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe309995000)
libcurl.so.4 => /opt/bin/../lib/libcurl.so.4 (0x00007fe30971c000)
libxml2.so.2 => /opt/bin/../lib/libxml2.so.2 (0x00007fe3093c5000)
libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007fe308f6e000)
libssl.so.10 => /lib64/libssl.so.10 (0x00007fe308cff000)
libzstd.so.1 => /opt/bin/../lib/libzstd.so.1 (0x00007fe308a75000)
liblzma.so.5 => not found
libdeflate.so.0 => /opt/bin/../lib/libdeflate.so.0 (0x00007fe308864000)
libblosc.so.1 => /opt/bin/../lib/libblosc.so.1 (0x00007fe30863c000)
libexpat.so.1 => /opt/bin/../lib/libexpat.so.1 (0x00007fe30840b000)
libjpeg.so.62 => /opt/bin/../lib/libjpeg.so.62 (0x00007fe308164000)
libtiff.so.6 => /opt/bin/../lib/libtiff.so.6 (0x00007fe307ee1000)
libgeotiff.so.5 => /opt/bin/../lib/libgeotiff.so.5 (0x00007fe307cad000)
libpng16.so.16 => /opt/bin/../lib/libpng16.so.16 (0x00007fe307a7a000)
libLercLib.so => /opt/bin/../lib/libLercLib.so (0x00007fe307811000)
libnetcdf.so.18 => /opt/bin/../lib/libnetcdf.so.18 (0x00007fe3074de000)
libhdf5.so.200 => /opt/bin/../lib/libhdf5.so.200 (0x00007fe306e0c000)
libsz.so.2 => /opt/bin/../lib/libsz.so.2 (0x00007fe306bf7000)
libwebp.so.7 => /opt/bin/../lib/libwebp.so.7 (0x00007fe30698c000)
libsqlite3.so.0 => /opt/bin/../lib/libsqlite3.so.0 (0x00007fe306678000)
libpq.so.5 => /opt/bin/../lib/libpq.so.5 (0x00007fe30642c000)
libopenjp2.so.7 => /opt/bin/../lib/libopenjp2.so.7 (0x00007fe3061cf000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fe305f6d000)
libgeos_c.so.1 => /opt/bin/../lib/libgeos_c.so.1 (0x00007fe305d1a000)
libproj.so.25 => /opt/bin/../lib/libproj.so.25 (0x00007fe305771000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe305553000)
libdf.so.0 => /opt/bin/../lib/libdf.so.0 (0x00007fe3052a2000)
libmfhdf.so.0 => /opt/bin/../lib/libmfhdf.so.0 (0x00007fe305078000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe30c567000)
libnghttp2.so.14 => /opt/bin/../lib/libnghttp2.so.14 (0x00007fe304e4d000)
liblzma.so.5 => not found
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fe304c01000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fe30491d000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fe304719000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fe3044e8000)
liblzma.so.5 => not found
liblzma.so.5 => not found
libhdf5_hl.so.200 => /opt/bin/../lib/libhdf5_hl.so.200 (0x00007fe3042c6000)
libgeos.so.3.12.1 => /opt/bin/../lib/libgeos.so.3.12.1 (0x00007fe303d38000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fe303b29000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fe303925000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fe30370f000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fe3034e8000)
Edit: in the meantime, I switched to python and it works.
from docker-lambda.
Related Issues (20)
- GDAL 3.2.2 is out HOT 1
- GDAL branch change to `main` HOT 1
- Can we get DXF vector drivers? HOT 4
- Need support for python3.9 HOT 2
- PROJ_DATA to replace PROJ_LIB environment variable HOT 1
- Trouble using lambda layers (python 3.9 / gdal 3.5) HOT 3
- Geopandas and Fiona success!
- zip not found in the docker image HOT 1
- Release a version for GDAL 3.6 HOT 1
- "RUN yum install -y gcc gcc-c++" adds 500MB to image but is unused at runtime HOT 2
- ERROR 1: PROJ: SQLite3 version is 3.7.17, whereas at least 3.11 should be used HOT 8
- Is latest 3.6 compiled with parquet / arrow enabled? HOT 4
- GDAL 3.7 HOT 4
- Invoking lambda always throws "No module named 'osgeo' " HOT 1
- LAMBDA_TASK_ROOT instead of PACKAGE_PREFIX? HOT 2
- Lambda container images HOT 1
- Image publication for other linux 2 containers
- RPATH modification rationale? HOT 1
- libsqlite3 linking broken on 3.8 images HOT 4
- Failed building wheel for GDAL - missing `gcc` when building docker for Py3.11 + GDAL 3.6 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-lambda.