planetary-social / planetary-graphql

graphql endpoint to ssb-db2 secure scuttlebutt server

planetary-graphql

This is a graphql server that is designed to complement a room-server. When properly configured it will provide a GraphQL API allowing you to query:

• details about the room (name, notices)
• membership of the room
• posts made by room members

The data provided by this API conforms to the [ssb-public-web-hosting-spec]

Run locally (Development)

npm install
npm run dev

Deploy

Digitial Ocean setup

1. spin up a droplet, adding SSH keys
2. assign a "reserved IP address" (static address)
3. ssh into the droplet, e.g.

   ssh [email protected]

4. install node
   • install nvm
   • nvm install 16
     • (you may need to log out and in for nvm to be registered)
5. install planetary-graphql

   git clone https://github.com/planetary-social/planetary-graphql.git
   cd planetary-graphql
   npm i

6. set up environment variables

   cp .env.template .env

   • you will need to edit this file to make sure the details are correct`
     • these environment variables are required for room related API:
       • ROOM_KEY - ssb id of the room
       • ROOM_HOST - the hostname of the room
       • ROOM_URL - the URL the room is being served at. This should be an absolute URL. E.g. https://localhost:3000 or https://civic.love
       • MAGIC_TOKEN - the token used to bypass invites for the go-ssb-room server
7. install pm2 (process manager)

   npm install pm2 -g
   pm2 startup

8. start the process!

   npm start

9. check the server is live using your browser
   • visit e.g. http://157.230.72.191:4000/graphql (assume default PORT)

Other useful commands:

npm run stop          // close the pm2 process does
pm2 restart graphql   // restart the graphql process
pm2 monit             // detailed monitoring for pm2 processes

Pair with a Room Server

You will need to add this peer as a member of the room (so that it can poll the room for updates).

1. open the secret file at DB_PATH/secret (default: ./db/secret)
2. copy the id from this file
3. in the room-server, add this id as a member

Health checks

If using pm2, you can run pm2 list to see the cpu/mem/uptime of your server.

Checking disk usage in the planetary-graphql folder: bash du -h db/db2 ls -lh db/db2/log.bipf

Update

cd planetary-graph
git pull origin master
npm install
pm2 restart all

ssb-public-web-hosting-spec: https://github.com/ssbc/ssb-public-web-hosting-spec

planetary-graphql's Issues

getThreads

• Write the query to getThreads by members

For issue: planetary-social/rooms-frontend#128

Debug why graphql server isnt connecting to the room

#41

Look into why graphql.planetary.pub has stopped syncing

The pubs data is out of date, indicating it isnt syncing properly

getThread query

Write a query to get a single thread by messageId

Add check for out of date syncing for planetary-graphql servers

At the moment it seems that often the planetary-graphql server often gets stuck and stops syncing content. We need to create some way of monitoring what the current state of thing is so we can SEE when it's blocked and also track down why it's happening.

Timestamps are null?

Fix bug where timestamps are coming back from graphql as null

Investigate server performance on maxed out droplet

At the moment, we are running the graphql server in production on a server with 2gb of memory, and it is using 100% of the cpu's resources when doing initial loads. We had increased this server fromm its original 1gb memory because of this performance issue. What happens if we increase the server's power to something maximal, like 50gb.

Run an experiment to load a space like planetary.name (with its hundreds of members) on a box with maximal resources, and see if graphql still hits performance issues. This is to help us figure out either the minimum cpu size required by our server, or if there's another issue with it's memory and performance that needs investigating.

Thread pagination

Need to be able to paginate threads.

First pass solution:

• get all threads, and paginate the chunk requested from the front end

Second pass: (later)

• solve this problem further down in the stack: ssb-threads

get alias by feedId

We need to be able to load the alias of a room member by feedId

mapMessages refactor

Mix wants to pull it into the thread - loading logic so it's impossible to accidentally forget to do this.
Might rename "publicWebMessages" or something

Cant get the room when there are no notices

Getting an error when connecting to a room that has no notices... I think

index.e40c406c.js:93 Uncaught (in promise) Error: Cannot read property 'pinned_notices' of undefined
    at new t (index.e40c406c.js:93:158)
    at index.e40c406c.js:94:27504
    at h (index.e40c406c.js:68:8461)
    at index.e40c406c.js:68:8387
    at new Promise (<anonymous>)
    at Object.then (index.e40c406c.js:68:8354)
    at Object.next (index.e40c406c.js:68:8471)
    at Lw (index.e40c406c.js:68:1919)
    at O_ (index.e40c406c.js:68:2329)
    at t.next (index.e40c406c.js:68:2846)

thread pagination

Update getThreads graphql query with pagination

Use the room key to get the multiserver address

Use shorter uris for following

The current way we open peoples profiles in Planetary, with a code like this:

ssb:experimental?action=consume-alias&roomId=%40HGwGdPc0R8HG%2Bik9%2BCOgldeMq78sQ9O5adH9qlcNqBo%3D.ed25519&userId=%40UsApPEhMpZaoRzoT6PfWcBct5vOaHXntpndwAbTw3po%3D.ed25519&multiserverAddress=net%3Alocalhost%3A8008%7Eshs%3AHGwGdPc0R8HG%2Bik9%2BCOgldeMq78sQ9O5adH9qlcNqBo%3D

results in a complex QR code like this:

making it harder to scan.

• need to add support to planetary for different types of uris: https://github.com/ssbc/ssb-uri-spec

Change scalar type for "timestamp"

Problem:

When querying timestamp, you get an error:

Int cannot represent non 32-bit signed integer value: 1663879111572

Possible Solutions

• use a GraphqlDate scalar
• OR use Float scalar

Display Room Notices

Room notices a.k.a the description (i think?)

• extend the getRoom query to also return room notices/description

Update README.md

eager blob rendering

Thread replies pagination

Paginate the responses on a thread. Probably need to utilize ssb-threads threadMaxSize. This may not be needed

Ignored by

• write a resolver to get:
  • ignored by count
  • ignored by profiles

Graphql Server not connecting to room

Latest changes from #39 are preventing the graphql server from connecting to the room.

Steps to reproduce:

1. Pull down the latest changes
2. Install dependencies
3. Start the development server npm run dev

Expected:

• it to start and connect to the room

Actual:

• getting an error in the console:

server error, from net:127.0.0.1:50918~shs:
Error: shs.server: client hello invalid (phase 3). they dailed a wrong number - they didn't have our public key
    at abort (/Users/planetary/planetary-social/planetary-graphql/node_modules/secret-handshake/protocol.js:82:45)
    at Object.cb (/Users/planetary/planetary-social/planetary-graphql/node_modules/secret-handshake/protocol.js:96:20)
    at drain (/Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:39:14)
    at more (/Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:55:13)
    at /Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:66:9
    at /Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:20:7
    at drain (/Users/planetary/planetary-social/planetary-graphql/node_modules/stream-to-pull-stream/index.js:126:18)
    at Socket.<anonymous> (/Users/planetary/planetary-social/planetary-graphql/node_modules/stream-to-pull-stream/index.js:135:5)
    at Socket.emit (events.js:400:28)
    at addChunk (internal/streams/readable.js:293:12) {
  address: 'net:127.0.0.1:50918~shs:'
}
failed to connect to room Error: shs.client: server hung up when we sent hello (phase 3).
    at clarify (/Users/planetary/planetary-social/planetary-graphql/node_modules/clarify-error/lib/index.js:11:24)
    at abort (/Users/planetary/planetary-social/planetary-graphql/node_modules/secret-handshake/protocol.js:37:45)
    at Object.cb (/Users/planetary/planetary-social/planetary-graphql/node_modules/secret-handshake/protocol.js:52:26)
    at drain (/Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:43:23)
  Error: stream ended with:64 but wanted:80
    at drain (/Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:43:26)
    at /Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:63:18
    at /Users/planetary/planetary-social/planetary-graphql/node_modules/pull-reader/index.js:20:7
    at drain (/Users/planetary/planetary-social/planetary-graphql/node_modules/stream-to-pull-stream/index.js:126:18)
    at Socket.<anonymous> (/Users/planetary/planetary-social/planetary-graphql/node_modules/stream-to-pull-stream/index.js:143:5)
    at Socket.emit (events.js:412:35)
    at endReadableNT (internal/streams/readable.js:1333:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21) {
  address: 'net:127.0.0.1:8008~shs:'
}

Other things I tried:

• deleting my database and start from a fresh one. I still saw this same error for the new database
• rewinding the room server to a commit where it was working, to confirm it wasnt the room. Confirmed it wasnt the room server
• rewinding to the commit before this PR, i no longer saw this issue
• readded the graphql server as member when i deleted the database
• printing out the multiserver address, it looked correct to me

Add resolver for room invite code

• generate a magic token for the room
• pass the magic token to the graphql server through an environment variable
• extend room typeDef to include room.inviteCode
  • generate the invite code using the magic token and room details

Load alias by feedId

Context: When loading a profile by feedId, we want to load their aliases in the room as well.

Possible solutions:

• extend go-ssb-room to give us this information
• OR we could just use ssb to get room/alias type messages to get the aliases but this relies on the ssb server behind graphql having up to date alias information from the room (is that okay?)

TODO:

• extend graphql for profile.aliases to get alias by feedId
• extend graphql for profile.ssbURI to use an alias found (using get alias by feedId
• get alias by feedId (from either go-ssb-room OR room/alias messages

Considerations:

• multiple aliases in a single room

Create health API

add a graphql query to get the health of the server

Graphql Server follows room members

ssb-uri for members without an alias