arweaveteam / arweave-js Goto Github PK
View Code? Open in Web Editor NEWBrowser and Nodejs client for general interaction with the arweave protocol and gateways
License: MIT License
Browser and Nodejs client for general interaction with the arweave protocol and gateways
License: MIT License
test for external wallet is failing as permissions not given before signing.
it("should create and sign transactions using external implicit wallet",
Need to:
Current implementation for transactions.verify throws an error on ID/signature integrity check, then returns a boolean value on the signature verification.
This should be changed so it does one or the other as calling code currently needs to have a catch block and check the return value.
https://github.com/ArweaveTeam/arweave-js/blob/master/src/common/arweave/transactions.ts#L110
public async verify(transaction: Transaction): Promise<boolean>{
...
if (transaction.id !== expectedId) {
throw new Error(
`Invalid transaction signature or ID! The transaction ID doesn't match the expected SHA-256 hash of the signature.`
);
}
/**
* Now verify the signature is valid and signed by the owner wallet (owner field = originating wallet public key).
*/
return this.crypto.verify(
transaction.owner,
signaturePayload,
rawSignature
);
}
In the README.md
it's forgotten await
before arweave.createTransaction
.
The library computes a data_root even when there is no data. A transaction with data_size=0 and a non-empty data_root is invalid.
ArweaveTeam/arweave#105 introduced support for multiple transactions given posted transactions reference the last_tx
pulled from /tx_anchor
. This package needs to pull the tx from the anchor and use that when posting transactions to full support multiple transactions per block.
The word separately under Transactions
Integrating a chainable, function based library for interfacing with Arweave's GraphQL library would be a great feature to have for the project. The following are a proposed implementation for interfacing with GraphQL.
const arweave = Arweave.init({ ... });
const result = await arweave.query({ ...options }); // Initializes a new query
arweave.query({
// Default has the full GraphQL object
output: `{
cursor
node {
...
}
}`,
});
arweave
.query()
.tag({
name: 'name',
values: ['values'],
sort: 'ASC',
})
.tag({ ... });
arweave
.query()
.after('cursor')
.limit(10);
arweave
.query()
.sort('ASC');
arweave
.query()
.blocks({ min: 0, max: 10 });
arweave
.query()
.owners([...addresses]);
arweave
.query()
.recipients([...addresses]);
arweave
.query()
.blockId([...]);
it takes a very long time to retrieve tx data by chunk.
unit tests taking extra long times to conclude also.
can the running time be improved?
When calling the uploader.uploadChunk()
function, this error is produced. This is following the tutorial code for uploading a file using chunks.
Chrome: 94.0.4606.81
AR Connect Extension: 0.3.5
AR Balance in Wallet: 4.51 AR ($233)
arweave: 1.10.18
react-scripts: 4.0.3
react: 17.0.2
typescript: 4.1.5
I'm building a decentralised version of YouTube, our platform uploads thumbnails and MP4 videos (multiple GBs in size) straight to ARWeave using the arweave-js
package running in the browser in our React web app. I want to go live in coming weeks and I am currently blocked due to this issue.
Any file I upload above 5MB causes await arweave.transactions.sign(transaction)
to block for around 5 seconds, then throws the following Unhandled Exception in the console:
Uncaught (in promise) Error: Message length exceeded maximum allowed length.
This is clearly not the actual error, I've debugged the internal source code of arweave-js
and I cannot find where things are failing.
const transaction = await arweave.createTransaction({ data: file })
transaction.addTag('Content-Type', 'video/mp4')
await arweave.transactions.sign(transaction)
file
is an ArrayBuffer
of the MP4Really hope you all can help :)
Hello,
I know that you can get the content using arweave.transactions.getData(txn_id), but I want to get the content type to call different handlers, For example, if it is a image/png, I will call the picture handler.
I used the arbundle to upload content, I tried to get the Tags to get the content-type, but only got:
Bundle-Format : binary
Bundle-Version : 2.0.0
Can you help meοΌ
Thanks
Currently one needs to get the transaction first and then get the data. It would be cool if there would be a single method to just get the data based on the transaction id.
When I create a transaction with .createTransaction()
and specify a reward e.g reward: 1152971595
; I can .transactions.sign()
but when I .transactions.post()
a get response object like : { status: 400, data: "Invalid JSON." }
If I cast the reward to a string the broadcast is successful.
Since Number
is a valid JSON value format it would be nice to have a better error message, and even better if the validation would occur during
.createTransaction()
Description
I was just trying out this package with Angular and when I installed the package it gave me a compilation error after the code example that is on your README.
Code
Proposed solution
So I went to see the source code and changed the static init: (apiConfig: ApiConfig) => Arweave; to static init: (apiConfig:? ApiConfig) => Arweave; and everything worked as intended.
I don't know if this is the expected behavior but I don't think it is because you refer in your documentation that the init function can be called without parameters and it seems to be working if you change the parameter to optional. Also, you should change your docs in the npm web section to import Arweave from 'arweave/web' instead of import Arweave from 'arweave'.
This would make it possible to include them in sourcemaps from bundlers and fix this:
β οΈ Could not load source file "../../../../src/common/lib/error.ts" in source map of "../../arweb/node_modules/arweave/web/lib/error.js".
β οΈ Could not load source file "../../../../src/common/lib/transaction.ts" in source map of "../../arweb/node_modules/arweave/web/lib/transaction.js".
β οΈ Could not load source file "../../../../src/common/lib/utils.ts" in source map of "../../arweb/node_modules/arweave/web/lib/utils.js".
β οΈ Could not load source file "../../../src/web/index.ts" in source map of "../../arweb/node_modules/arweave/web/index.js".
β οΈ Could not load source file "../../../src/common/common.ts" in source map of "../../arweb/node_modules/arweave/web/common.js".
β οΈ Could not load source file "../../../../../src/common/lib/crypto/webcrypto-driver.ts" in source map of "../../arweb/node_modules/arweave/web/lib/crypto/webcrypto-driver.js".
β οΈ Could not load source file "../../../src/common/ar.ts" in source map of "../../arweb/node_modules/arweave/web/ar.js".
β οΈ Could not load source file "../../../src/common/wallets.ts" in source map of "../../arweb/node_modules/arweave/web/wallets.js".
β οΈ Could not load source file "../../../src/common/transactions.ts" in source map of "../../arweb/node_modules/arweave/web/transactions.js".
β οΈ Could not load source file "../../../src/common/silo.ts" in source map of "../../arweb/node_modules/arweave/web/silo.js".
β οΈ Could not load source file "../../../../src/common/lib/api.ts" in source map of "../../arweb/node_modules/arweave/web/lib/api.js".
β οΈ Could not load source file "../../../src/common/network.ts" in source map of "../../arweb/node_modules/arweave/web/network.js".
Greetings, is there way to show the cost of a transection before posting with arweave-js? Thanks!
Currently that node support a webcrypto API
So, we can get rid of the usage of crypto-browserify package
Hello in the buildLayers function, the very first if statement checks that the node.length is less than 2. Only 1 and 0 is less than 2 and yet the call to hashBranch takes two node parameters. If the length is 0 this would fail. If the length is 1 this would also fail because hashBranch takes two non-nullable parameters.
Next issue is in hashBranch itself. First an if statement runs to see if right node is null and then if true it returns a casted BranchNode. This is an issue because the function generateTransactionChunks first calls generateLeaves, which of course returns leaves, and then passes those leaves to the buildLayers function. If the leaves nodes is less than two then buildLayers call to hashBranch goes to the if statement at the top which casts the node into a BranchNode, but the node is not a BranchNode it is a LeafNode. Later in other parts of code if this node is called as a BranchNode it will fail.
Requesting: https://arweave.net:443/tx
{
status: 500,
statusText: 'Internal Server Error',
If submit transaction is unsigned then throw error?
This is a suggestion. I am building an arweave swift based api. In my transactions I may include data. There was an issue in my Merkle code which caused the data_root to be invalid. However the transaction still went through without error. I think this is wrong as a transaction that contains data should fail entirely if the data cannot be included.
The saltLength parameter of the sign method is invalid.
And why does the verify method, which validates both saltLength 0 and saltLength 32, not accept the saltLength argument?
I'm using ArweaveJS in Node.
I did the initialization like this:
const Arweave = require('arweave/node');
const arweave = new Arweave.init({
'host':'127.0.0.1',
'port':1984,
'protocol':'http'
});
But I keep getting this error:
All i'm trying to do is follow what is given in README to get started with arweave js. What I am doing wrong and How may I rectify it?
The getPrice()
api should accept a bigint to allow getting prices for transactions larger than 2GB.
arweave-js/src/common/transactions.ts
Line 44 in 1aa25ed
It's no longer recommended and it's not supported in service workers, so it makes sense to move to window.fetch
We need these two util functions, to make it easier to assemble and store uploader data. Currently, neither JSON.stringify
nor serialize-javascript
can restore the uploader in its entirety, causing the continuation upload to fail.
Hey, when I import Arweave from arweave-js with Svelte/Sapper, I get the errors pasted below.
My code:
import Arweave from "arweave/web"; // the compiler fails here
const client = new Arweave({
host: "arweave.net",
port: 443,
protocol: "https",
timeout: 20000,
});
Client (Svelte):
Circular dependency: node_modules\arweave\web\common.js -> node_modules\arweave\web\transactions.js -> node_modules\arweave\web\lib\transaction.js -> node_modules\arweave\web\lib\deepHash.js -> node_modules\arweave\web\common.js
Server (Sapper with polka):
β server
Unexpected token (Note that you need @rollup/plugin-json to import JSON files)
1: {
2: "name": "axios",
^
3: "version": "0.18.1",
4: "description": "Promise based HTTP client for the browser and node.js",
internal/modules/cjs/loader.js:330
throw err;
^
Error: Cannot find module 'C:\Users\user\Desktop\Node\coinary-frontend\node_modules\arweave\index.js'. Please verify that the package.json has a valid "main" entry
at tryPackage (internal/modules/cjs/loader.js:322:19)
at Function.Module._findPath (internal/modules/cjs/loader.js:683:18)
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:953:27)
at Function.Module._load (internal/modules/cjs/loader.js:842:27)
at Module.require (internal/modules/cjs/loader.js:1026:19)
at require (internal/modules/cjs/helpers.js:72:18)
at Object.<anonymous> (C:\Users\user\Desktop\Node\coinary-frontend\__sapper__\dev\server\server.js:10:1)
at Module._compile (internal/modules/cjs/loader.js:1138:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
at Module.load (internal/modules/cjs/loader.js:986:32) {
code: 'MODULE_NOT_FOUND',
path: 'C:\\Users\\user\\Desktop\\Node\\coinary-frontend\\node_modules\\arweave\\package.json',
requestPath: 'arweave'
}
> Server crashed
Full log from terminal:
src\stores\keyfileStore.js changed. rebuilding...
β’ client
Circular dependency: node_modules\arweave\web\common.js -> node_modules\arweave\web\transactions.js -> node_modules\arweave\web\lib\transaction.js -> node_modules\arweave\web\lib\deepHash.js -> node_modules\arweave\web\common.js
Circular dependency: node_modules\arweave\web\common.js -> node_modules\arweave\web\transactions.js -> node_modules\arweave\web\lib\transaction.js -> node_modules\arweave\web\lib\deepHash.js -> C:\Users\user\Desktop\Node\xy-frontend\node_modules\arweave\web\common.js?commonjs-proxy -> node_modules\arweave\web\common.js
Circular dependency: node_modules\arweave\web\common.js -> node_modules\arweave\web\transactions.js -> node_modules\arweave\web\lib\transaction.js -> node_modules\arweave\web\lib\merkle.js -> node_modules\arweave\web\common.js
β service worker (1.2s)
β server
Unexpected token (Note that you need @rollup/plugin-json to import JSON files)
1: {
2: "name": "axios",
^
3: "version": "0.18.1",
4: "description": "Promise based HTTP client for the browser and node.js",
internal/modules/cjs/loader.js:330
throw err;
^
Error: Cannot find module 'C:\Users\user\Desktop\Node\xy-frontend\node_modules\arweave\index.js'. Please verify that the package.json has a valid "main" entry
at tryPackage (internal/modules/cjs/loader.js:322:19)
at Function.Module._findPath (internal/modules/cjs/loader.js:683:18)
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:953:27)
at Function.Module._load (internal/modules/cjs/loader.js:842:27)
at Module.require (internal/modules/cjs/loader.js:1026:19)
at require (internal/modules/cjs/helpers.js:72:18)
at Object.<anonymous> (C:\Users\user\Desktop\Node\xy-frontend\__sapper__\dev\server\server.js:10:1)
at Module._compile (internal/modules/cjs/loader.js:1138:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
at Module.load (internal/modules/cjs/loader.js:986:32) {
code: 'MODULE_NOT_FOUND',
path: 'C:\\Users\\user\\Desktop\\Node\\xy-frontend\\node_modules\\arweave\\package.json',
requestPath: 'arweave'
}
> Server crashed
I need Transaction class to be exported.
A malicious or erroneous site may spend user's AR tokens in unlimited quantities by uploading big data blobs.
Additionally the user may mistakenly select a big file and spend more tokens than he or she intended.
Solution: Ask before spending money like MetaMask.
Use Uint8Array
types instead of Buffer
so typescript doesn't require @types/node
when using arweave/web
module.
It would be cool if there would be a parsable language for arql so I could write one as a string instead of using nested objects
I've written some code down here: https://gist.github.com/mkg20001/42a252f5fdc0a27d4ff3ed055edaf2cb
Implemented examples: & (= a 'test') (= b 'test')
and and(equals(a, 'test'), equals(b, 'test'))
Since both would use the same lexer (the lexer is actually not written yet), they could be both supported simultaniously.
Both token parsing functions return arql queries.
So far I've only implemented them to support and
and equals
How should those query languages be implemented in arweave?
So I am trying to save and get the image of 415kb. That works fine. I get status 200.
When I try to get data, I use getData function and I get back empty string.
Just replacing that image with 11kb image for example, works fine.
Is there a different method I need to use? From doc it says it supports larger uploads...
I use testweave-sdk. Could that be the issue that it doesn't return it?
Transactions from createTransaction and createSiloTransaction should be signed by default as they can be signed again if the consuming application wants to modify the contents, add tags etc before submitting.
Currently only a simple key/value search is possible, but we should support more complex queries with multiple tags.
Current interface
https://github.com/ArweaveTeam/arweave-js/blob/master/src/common/arweave/transactions.ts#L71
search(tagName: string, tagValue: string): Promise<string[]>;
This method wraps the key and value into a single aqrl expression like this
{
"op": "equals",
"expr1": "Content-Type",
"expr2": "text/html"
}
New requirement
An example of a more complex query that needs to be supported would look like this
{
"op": "and",
"expr1": {
"op": "or",
"expr1": {
"op": "equals",
"expr1": "Content-Type",
"expr2": "text/html"
},
"expr2": {
"op": "equals",
"expr1": "Content-Type",
"expr2": "application/bdoc"
}
},
"expr2": {
"op": "equals",
"expr1": "User-Agent",
"expr2": "ArweaveDeploy/1.0.5"
}
}
Expressions can be nested and use combinations of and
,or
, and equals
operators. IIRC there isn't a limit to nesting depth but I'll let @samcamwilliams confirm or clarify if there's a practical limit to consider.
Hello
I have added the following this link https://github.com/ArweaveTeam/arweave-js#create-a-wallet-to-wallet-transaction. Below my code:
try{
let key = req.body.key;
let transaction = await arweave.createTransaction({
target: req.body.address,
quantity: arweave.ar.arToWinston(req.body.amount)
}, key);
await arweave.transactions.sign(transaction, key);
console.log(transaction)
res.send({'data' : transaction});
}catch(e) {
return next(e)
//res.send({'error': '404'});
}
And the result when I send a post request:
"data": {
"format": 2,
"id": "",
"last_tx": "rlQ0NiHM_pflTufixBksNcIIaN9Cs3XZF3a-cJZ-FRYmojg589kaBlUXFavm1PI4",
"owner": "uO8Naab8ZgT8zBauZL7lmQ5tkC53MUa3TPU52Ld9dkXjELHHnZJiq6dns_PGeRYMnGl6810boqqgS1TE8D26BC7ZxD_oYqaFby7313hsQx_IN7q1oI7dHlO74kYR-2E0MMidSZ8lJvtrm92g7okI6EwU3g6XoRxTFgcRmj81aDqV-PmYHeGmNX1rhyTQ4O1I8AbBrLBBF_DeAcaMv1uRfHyiWoIIGiaqNEiP0fd7Wy1S2_GfaG-WtPZWJQw1K9xmryfLHhBewWn_DoDrygofStH3AMs-UIL0PU2BgzVawmcxNLGHvZFQ06jRJQ-GrU2-t5fQV9A--N8wIPNdIo0Sb51PWCbFKKosfFoQWoLpN-2ruZDJmHUlvB054CVCwG4kTPWYO4SkGhRuBJ79XAVa_gon5z-adHkLZcaZQdSvwma3B76rXDc_4c1ZT1-N7GlKc5ZEnRVrRvReZD--IH2rVMlxJzXPVWL6LsD-__9KdvGKDVX7NHG9h3lEqHmggB3dqKClXBYqIqq8o7__oB6BXvHiNB0dc1xQQjgN2Kcek_q9ag7qVAUgi6laPzYBhxf4g63kV4-jBbMnMwNNeDKHSiAsV1wcjoSnMOs-X9R2x3XVsVaos_xwqRMKTtiRggwu8G0GVHrrl5v9d4_rjvZxi8W6Ur-Jft1ee2BkFlX1pbc",
"tags": [],
"target": "PhXR4tD2WYMeCsoDx2-8abKhx8zOXYIA3yktOiWkmxA",
"quantity": "10000000000",
"data": "",
"data_size": "0",
"data_root": "",
"reward": "11416836907",
"signature": ""
}
}
but it seem the transaction have not sent to the address PhXR4tD2WYMeCsoDx2-8abKhx8zOXYIA3yktOiWkmxA
Doing
arweave.transactions.getData(
"8tqZAMa4w_tlzkaNba8lyo2-gY6qQ8TYk_6A7yxOzfE",
{ decode: true, string: true }
);
results in TX_NOT_FOUND
error even though tx exists: https://arweave.net/tx/8tqZAMa4w_tlzkaNba8lyo2-gY6qQ8TYk_6A7yxOzfE
Both https://github.com/ArweaveTeam/arweave-js#create-a-wallet-to-wallet-transaction and https://github.com/ArweaveTeam/arweave-js#create-a-data-transaction instantiate a variable named key
but then use a variable named jwk
as the second parameter in the createTransaction
function. I imagine this is a typo but is potentially confusing.
Currently, importing Arweave-JS in a browser based project that uses Webpack 5 will result in a build-time error of the below sort:
ERROR in ./node_modules/arweave/web/lib/merkle.js 9:15-30
Module not found: Error: Can't resolve 'util' in '/home/jim/development/monkeys1/node_modules/arweave/web/lib'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
This can be worked around at present by adding the below to webpack.configs.js
:
module.exports = {
...
plugins: [
//Polyfill needed nodejs plugins for Arweave-js
new webpack.ProvidePlugin({
process: 'process/browser',
}), new webpack.ProvidePlugin({
util: 'util/',
}),
...
}
and then npm i --save-dev util process
to add these two node-js polyfill libraries to a given project.
Ideally, these dependencies would be resolved by browser-compatible imports within arweave/web//lib/merkle
The following query
{
transaction(
id: "wVK2lQ3U6wm7fXjjRwKgKTs4LPi7TqWn9tjisD7gIxI"
) {
id
owner { address }
block { timestamp }
}
}
returns
{
"data": {
"transaction": {
"id": "wVK2lQ3U6wm7fXjjRwKgKTs4LPi7TqWn9tjisD7gIxI",
"owner": {
"address": "aIUmY9Iy4qoW3HOikTy6aJww-mM4Y-CUJ7mXoPdzdog"
},
"block": null
}
}
}
"block": null
This happen for any query of a single transaction
As a programmer and a normal person who uses the capabilities of Arweave, I would like to be able to easily sign data, including graphic or video files.
Currently, there is no function that is responsible for just signing the data (at least from what I have noticed) but only signing the transaction.
I suggest creating a special class where it will be possible to sign data in text format as well as files.
let write = await arweave.signatures.create({
key: 'TheSignerPrivateKey',
address: 'the address of the person who will be able to read the data',
data: ['./image.png', './video.mp4', 'document.pdf', 'buffer or data string']
});
let read = await arweave.signatures.read({
key: 'TheReceiverPrivateKey',
data: 'Hash of previously signed data',
path: 'Data storage path (if files)'
});
I'm starting to work on it and PR will appear soon. If someone wants to help me, I will always be grateful for TIP to the addresses below :D I invite you to discussions and suggestions.
ETH: 0xCf44E2Ae6f68a0412D52ABdb6B41A4A59e2E3306
AR: KTcbsn6vg1Ud5ZDSKgbYisc7SVBsalkJprsQtGNjVN8
arweave-js/src/common/transactions.ts
Line 347 in 92d690a
When I pass the transaction ID to the getUploader method, the data gets the transaction information through the get
API, but the returned upload data without the chunks field.
This causes the chunks check to not pass fromSerialized
, so it will throw an error named Transaction chunks not prepared
i installed the npm module and imported it:
let Arweave = require('arweave/web');
const arweave = Arweave.init({
host: 'arweave.net',
port: 80,
protocol: 'https'
});
but i get an error that in the npm arweave/web module, window is not defined
arweave/web/index.js:14
window.Arweave = common_1.default;
^
ReferenceError: window is not defined
create a publish workflow (and maybe disable manual publish?) so as to publish only the fully tested build
arweave-js/src/common/network.ts
Line 4 in 80c4585
Is this supposed to be
network
instead of [misspelled, misnamed]
hosnetworkt
?
ARQL currently only has a few operators. It would be nice to expand the operator toolkit with the following functionality.
limit: number
This would limit the amount transaction ids returned.
skip: number
This would skip over n
amount of transaction ids and return anything after.
sort: { tag: string, direction: enum[asc|desc] }
This would sort the transaction ids based on the associated tag values.
I understand that sort
might not be feasible given that indexing tags in this way could potentially be extremely costly. However, I think both the limit
and skip
functions are totally reasonable, and would improve the ARQL experience.
Update
Just chatted with JY
on the Solana Discord and it seems that there will be full GraphQL functionality for Arweave soon. These requests I have are probably moot given how GraphQL would pretty much solve all my problems.
When I run this code as http://zon.localhost/test.html the browser console shows that Arweave tries to access wrong URLs like:
http://zon.localhost/tx_anchor
<html>
<head>
<script src="https://unpkg.com/arweave/bundles/web.bundle.min.js"></script>
<script>
const arweave = Arweave.init();
function test() {
arweave.wallets.generate().then(async (key) => {
let transaction = await arweave.createTransaction({
data: "x",
}, key);
await arweave.transactions.sign(transaction, key);
const response = await arweave.transactions.post(transaction);
});
}
window.addEventListener('load', test);
</script>
</head>
<body>
<p>Test</p>
</body>
</html>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.