- Create an application that interacts with Hedera Consensus Service (HCS)
- Use the Hedera SDK to create a new HCS Topic
- Use the Hedera SDK to publish/ subscribe messages to the HCS Topic
- Use the Hedera Mirror Node to retrieve historical messages from HCS Topic
- Use JSON Schema and hashes to validate messages retrieved from HCS
You will be building an app that submits messages to a HCS Topic. Each message looks something similar to this:
{
"type": "hcs-skill/v1",
"accountId": "0.0.1521",
"skillName": "Hello World - Create and fund account",
"userName": "bguiz",
"hash": "9c17fcc378e286b2d4bcf693110fd53252eb23144818df21d86f6cdbc1c931a4",
}
These messages will be verified using JSON-schema, and also verified by checking that for a hash match.
In the user interface of the application, only messages that pass verification will be displayed. Those that do not pass verification will simply be filtered out.
To do all of this, you will make use of the following Hedera technologies:
- HAPIs via the Hedera SDK
TopicCreateTransaction
TopicMessageSubmitTransaction
TopicMessageQuery
- HTTP APIs via the Hedera Mirror Node
/api/v1/topics/{topicId}/messages
- Manual verification via Hashscan
You will also make use of the following 3rd-party technologies:
safe-stable-stringify
for data serialisationajv
for JSON Schema verificationexpress
for a simple web servernode:crypto
for hashing
Naturally, this workshop will mostly focus on the Hedera technologies.
Also, you should have the following set up on your computer โฌ
- POSIX-compliant shell
- For Linux & Mac: The shell that ships with the operating system will work. Either
bash
orzsh
will work. - For Windows: The shells that ship with the operating system (
cmd.exe
,powershell.exe
) will not work.- Recommended:
git-bash
which ships withgit-for-windows
. Install Git for Windows (Git for Windows) - Recommended (alternative): Windows Subsystem for Linux. Install WSL (Microsoft)
- Recommended:
- For Linux & Mac: The shell that ships with the operating system will work. Either
-
git
installed- Minimum version: 2.37
- Recommended: Install Git (Github)
- A code editor or IDE
- Recommended: VS Code. Install VS Code (Visual Studio)
- NodeJs +
npm
installed- Minimum version of NodeJs: 20
- Minimum version of
npm
: 9.5 - Recommended for Linux & Mac:
nvm
- Recommended for Windows:
nvm-windows
Check your prerequisites set up โฌ
Open your terminal, and enter the following commands.
bash --version
zsh --version
git --version
code --version
node --version
npm --version
Each of these commands should output some text that includes a version number, for example:
bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin22)
Copyright (C) 2007 Free Software Foundation, Inc.
zsh --version
zsh 5.9 (x86_64-apple-darwin22.0)
git --version
git version 2.39.2 (Apple Git-143)
code --version
1.81.1
6c3e3dba23e8fadc360aed75ce363ba185c49794
arm64
node --version
v20.6.1
npm --version
9.8.1
If the output contains text similar to command not found
, please install that item.
If the version number that is output is lower than the required versions, please re-install or update that item.
If the version number that is output is same or higher than the required versions, you have met the prerequisites! ๐
To follow along, start with the main
branch, which is the default branch of this repo. This gives you the initial state from which you can follow along with the steps as described in the tutorial.
git clone https://github.com/hedera-dev/hcs-skills.git
Alternative with `git` and SSH
If you have configured SSH to work with git
, you may wish use this command instead:
git clone [email protected]:hedera-dev/hcs-skills.git
Enter the hcs-skills
directory.
cd hcs-skills/
Install the dependencies using npm
.
npm install
Make a .env
file by copying the provided .env.sample
file. Then open the .env
file in a code editor, such as VS Code.
cp .env.sample .env
Then edit the .env
file.
OPERATOR_ID=
OPERATOR_PRIVATE_KEY=
Fill in the values for the OPERATOR_ID
and OPERATOR_PRIVATE_KEY
with those from your Hedera Testnet account. If you do not have one yet, you may obtain one from the Hedera developer portal.
Let's now check that the git repo, npm installation, and .env
file have been set up properly. Run:
node scripts/checkpoint-01-setup.js
This should produce an output that indicates whether each of the above steps has been performed correctly.
Sample output โฌ
git check:
OK!
npm install check:
OK!
.env file check:
OK!
You should see 3 OK!
outputs. If not, please revisit those steps which indicate an error.
Open schemas/hcs-skill--v1.schema.json
. Looking at the properties
object within the schema file, we see the following:
Schema file contents โฌ
"properties": {
"type": {
"const": "hcs-skill/v1"
},
"accountId": {
"type": "string",
"pattern": "^\\d+.\\d+.\\d+$"
},
"userName": {
"type": "string",
"minLength": 3,
"maxLength": 100
},
"skillName": {
"type": "string",
"minLength": 3,
"maxLength": 100
},
"hash": {
"type": "string",
"minLength": 64,
"maxLength": 64
}
},
This defines the validation rules for the skills objects that you will submit to the HCS Topic.
Open util/skill-verify.js
.
Look in the skillVerify
function, and within that find the comment // NOTE: Schema validation
.
At this point, the schema that we just looked at above has been read in, and compiled into a validator
object, using ajv
. Modify the statement to pass the obj
as a parameter to the validator
.
const isValid = validator(obj);
Stay within util/skill-verify.js
, and the skillVerify
function. Find the comment // NOTE: Custom validation
next.
At this point, the obj.hash
has been recomputed, and we need to check if this hash matches the hash that was originally in the object.
Create a version of the object with a hash added to it, by invoking the addHash
function.
const objWithUpdatedHash = addHash(obj);
Modify the condition of the if
statement to do the following.
obj.hash !== objWithUpdatedHash.hash
Let's now check that the skillVerify
function that we've just modified works as expected. Run:
node scripts/checkpoint-02-validation.js
This script attempts to validate 3 invalid objects, followed by 1 valid object.
This should produce an output that contains validation failures for the first 3 objects, and a validation success for the 4th (final) object.
Sample output โฌ
Note: Expect 3 validation failures, followed by 1 validation success.
Object #0:
Validation failure.
[
{
instancePath: '/type',
message: 'unsupported schema: some-other-type/v1'
}
]
Object #1:
Validation failure.
[
{
instancePath: '/skillName',
schemaPath: '#/properties/skillName/minLength',
keyword: 'minLength',
params: { limit: 3 },
message: 'must NOT have fewer than 3 characters'
}
]
Object #2:
81e3b744163404b9a0581547aa243fe2f12e86a1f72056512380a0429c76ef56
Validation failure.
[ { instancePath: '/hash', message: 'hash mismatch' } ]
Object #3:
Validation success!
Open util/skill-publish.js
.
Look in the skillPublish
function, and within that find the comment // NOTE: Add hash to message
.
Create an object named obj
by invoking addHash
on the skillData
object.
const obj = addHash(skillData);
The addHash
function is found within util/objects.js
. It simply serialises an object as a string, then hashes it using the sha256
hash function, and adds that hash to the original object. This has already been implemented for you, and no modification is necessary.
Stay within util/skill-publish.js
, and the skillPublish
function. Find the comment // NOTE: Verify message
next.
Before publishing the message, you will want to verify that it conforms to the required format of a skill object. To do so, invoke the skillVerify
function completed previously, by passing in obj
.
const validationErrors = skillVerify(obj);
This will return an array of validation errors, if there are any. The subsequent lines perform error handling, and blocks the message from being published if there is any error. This has already been implemented for you, and no modification is necessary.
Stay within util/skill-publish.js
, and the skillPublish
function. Find the comment // NOTE: Submit message to HCS topic
next.
Now you are finally ready to publish the message to the HCS topic. To do so, we send a TopicMessageSubmitTransaction
to the network. As its name suggests, it submits a message to a HCS topic. Pass in topicId
and hcsMsg
as properties of this transaction.
const topicMsgSubmitTx = await new TopicMessageSubmitTransaction({
topicId: topicId,
message: hcsMsg,
}).execute(client);
When .execute(client)
is invoked on the transaction, the transaction is cryptographically signed using your account configured in the .env
file, and then submitted to the network.
Let's now check that the skillPublish
function that we've just modified works as expected. Run:
node scripts/checkpoint-03-publish.js
This script attempts to publish a message that fails validation, and subsequently attempts to publish a message that passes validation.
This should produce an output that contains 1 rejection, followed by 1 successfully submitted message.
Sample output โฌ
Expect validation error:
[
{
instancePath: '/skillName',
schemaPath: '#/properties/skillName/minLength',
keyword: 'minLength',
params: { limit: 3 },
message: 'must NOT have fewer than 3 characters'
}
]
skill validation failed
Expect success status + hash:
Status { _code: 22 }
68b0f367570ea93073a34755ec66a647a9776a09151fae1bf7d97157ec321e7c
Open util/skill-subscribe.js
.
Look in the skillGetAll
function, and within that find the comment // NOTE: Mirror Node query of HCS topic
.
Ensure that the topicId
is within the URL, before using it in an HTTP request, like so:
const mirrorNodeUrl =
`https://testnet.mirrornode.hedera.com/api/v1/topics/${topicId.toString()}/messages`;
const fetchResponse = await fetch(mirrorNodeUrl);
This is a standard HTTP API, and can be queried without using the Hedera SDK. In fact, in this example we are using fetch
to do so. You can explore this API, and others related to HCS Topics, using the interactive docs available at Hedera Mirror Node REST API.
Stay within util/skill-subscribe.js
, and look in the skillSubscribe
function. Find the comment // NOTE: Subscribe to HCS topic
next.
In skillGetAll
you performed a one-off query to get all prior messages in the topic. In this skillSubscribe
function, we intend to do something different, which is to perform a persistent query to get all future messages in the topic -- in other words, a subscription.
To do so, create a new TopicMessageQuery
and invoke setTopicId
.
new TopicMessageQuery()
.setTopicId(topicId)
.subscribe(client, (msgBin) => parseSkill(msgBin.contents, 'utf8', callback));
Note that unlike other interactions via the SDK, which end with .execute(client)
, this one ends with .subscribe(client, callback)
. This is because this is still a query (as opposed to a transaction which can alter the state of the network), and because it is a subscription.
Let's now check that the skillGetAll
and skillSubscribe
functions that we've just modified work as expected. Run:
node scripts/checkpoint-04-subscribe.js
This script invokes skillGetAll
which queries all past messages on a HCS topic, and tallies how many of them are valid skill objects, and how many are not. The script also invokes skillSubscribe
, and then immediately after invokes skillPublish
(since skillSubscribe
only tracks new messages on a HCS topic), then finally tallies how many of them are valid skill objects, and how many are not.
This should produce an output that contains counts for valid and invalid messages for both skillGetAll
and skillSubscribe
.
Sample output โฌ
skillGetAll message counts:
{ validCount: 7, invalidCount: 2 }
Waiting 5s...
skillSubscribe message counts:
{ validCount: 1, invalidCount: 0 }
Waiting 5s...
If you re-run the same script immediately after, you will notice that the validCount
for skillGetAll
increments by 1. The other values remain the same. This is because the script publishes a new valid message to the HCS topic each time it is run for the purposes of testing.
skillGetAll message counts:
{ validCount: 8, invalidCount: 2 }
Waiting 5s...
skillPublish message counts:
{ validCount: 1, invalidCount: 0 }
Waiting 5s...
Open front/index.js
.
Look in the subExistingTopic
function, and within that find the comment // NOTE: Subscribe to topic
.
The intent of this function is to subscribe to a particular topic. Recall in "Step 11: Subscribe to HCS topic" that you implemented a skillSubscribe
function which invokes TopicMessageQuery
through the SDK. That skillSubscribe
function is exposed via the server API, and available at the path /api/v1/topic/subscribe/TOPIC_ID
.
To start the subscription within the application, invoke fetch
, and pass in the URL, substituting the Topic ID appropriately, like so:
const response = await fetch(
`/api/v1/topic/subscribe/${textInputTopicId}`,
{
method: 'GET',
headers: {
'Content-Type': 'application/json',
},
},
);
To see how the front end and back end work together, you may want to check out the following:
- In
back/server.js
, the API route handler forserver.get('/api/v1/topic/subscribe/:topicId', ...);
invokesback/subscribe-to-topic.js
, which in turn invokesutil/skill-subscribe.js
, where the SDK is used to perform the subscription. - Each time a message on the Topic is detected in
util/skill-subscribe.js
, it passes that onto the callback inback/subscribe-to-topic.js
namedonGetSkillCallback
. - The
onGetSkillCallback
emits a message on a web socket with the IDhcs-skill-TOPIC_ID
. - On the client, see the
updateSubscribedTopic
function withinfront/index.js
, wheresocket.on(data.socketId, onSocketHcsSkill);
listens for messages on a websocket with the IDhcs-skill-TOPIC_ID
. TheonSocketHcsSkill
function parses each message and displays it in the UI.
Now you have a working application built on top of Hedera Consensus Service!
๐๐๐
Possible stretch goals:
- Easy: Create a more detailed schema for specific use cases
- For example: https://github.com/OpenCerts
- Moderate: Investigate the use of
setSubmitKey
onTopicCreateTransaction
to create a private HCS topic. Use this in combination withThresholdKey
to create a whitelist of accounts that may submit to the private topic. - Hard: Modify
skillVerify
such that theaccountId
in the message matches the account that submitted the HCS transaction - Very hard: Modify system to enable multiple schemas for different object types being published to the same schema, and to allow them to reference each other
- For example: Define
person
andskill
as separate entities, where there is a many-to-many relationship between the two.
- For example: Define
- Create an application that interacts with Hedera Consensus Service (HCS)
- Use the Hedera SDK to create a new HCS Topic
- Use the Hedera SDK to publish/ subscribe messages to the HCS Topic
- Use the Hedera Mirror Node to retrieve historical messages from HCS Topic
- Use JSON Schema and hashes to validate messages retrieved from HCS
Apache-2.0