beakerbrowser / unwalled.garden

Schemas for a p2p social-media network built on the Dat Web.

unwalled.garden

Schemas and APIs for social applications on the Dat Web.

Join us on Freenode at #unwalled.garden.

How it works

• Every user has their own Dat website.
• Users publish statuses, comments, and other kinds of content on their sites.
• Users follow each other to sync their content.
• Follows are public, creating a social graph.

You can think of it as a souped-up RSS: users publish records as files on their sites, then sync the files regularly to receive updates. This is used to power news feeds, aggregators, comments sections, search engines, and more.

Visit the website to learn more.

Governance

The rules for governance are as follows:

1. Open issues or pull requests to discuss changes, problems, and ideas.
2. Pull requests should be made to the develop branch.
3. As unwalled.garden is a decentralized network, breaking changes are not allowed.
4. All PRs require the BDFL's final approval before merging.

The current BDFL is Paul Frazee.

The current reference implementation is Beaker. The develop branch will be merged into master when the reference implementation has been published with appropriate support for the changes.

unwalled.garden's Issues

Make an IRC channel

maybe #unwalled on irc.freenode.net

extending the schema to work with git

Hi, author of isomorphic-git here. I'm always thinking about how to use git for things. 🤷‍♂ I was thinking about the unwalled.garden file structure and it occurred to me... if it's just files, it doesn't have to use DAT does it? The same directory could be published using git couldn't it? (Please ignore why users would want to use git instead of dat... just accept it as an axiom momentarily)

---

How it works

• Every user has their own Dat website git repo.
• Users publish posts, comments, and other kinds of content on their sites git repo.
• Users follow each other to sync their content.
• Follows are public, creating a social graph.

You can think of it as a souped-up RSS: users publish records as files on their sites in their git repos, then sync fetch the files repos regularly to receive updates. This is used to power news feeds, aggregators, comments sections, search engines, and more.

---

If I'm thinking about this correctly, then everything in the following schemas is dat-agnostic and could work out-of-the-box:

• Bookmarks
• Comment
• Posts
• Reaction
• the /.data directory

and the following would work as long as non-dat URIs are allowed:

• Follows - it would need to be a list of git repos, not Dat sites.

and the last bit would....... require some heavy tweaking

• the /.refs directory - maybe I could represent them as git submodules

My Thoughts

If you are excited about this, I think I could make a proof-of-concept PWA - that works in any browser - that lets users create Posts (by pushing to Github) and read their friends Posts (by fetching from Github) in a matter of a few evenings.

If I did this - and we now had two clients using two protocols - would we want to make any changes to the Follows schema? At the time I'm writing this, it is a single array of URI strings. The most obvious change would be to add some text saying something like "URIs can have protocols other than Dat. Implementations should ignore any URIs with protocols they do not support. The interpretation of non-Dat protocols is implementation specific and not covered in this specification".

Suggestion for Follows Schema

Instead of having the follows schema only consist of urls, it could also include a nickname we choose to give the person we're following. If several urls have the same name then this would help the user distinguish between the different people they follow who may have the same preferred name.

Defs

• preferred name: the name you give yourself (I'd call myself "ghiliweld")
• nickname: the name someone who follows you might give you to help them identify you (someone might prefer to just call me "ghili")

Example

{
  "following": [
    { 
      "url" : "dat://43dfc9f23fdded8cc7c01c71c0702a0529130af0258e7fb30bf5a0a3f73d69b3",
      "nickname": "john"
    }
  ]
}

Minor issue on GitHub readme (links = 404)

Just a minor housekeeping issue:
https://github.com/beakerbrowser/unwalled.garden

On the README.md

Governance

The rules for governance are as follows:

Open **issues** or **pull requests** to discuss changes, problems, and ideas.

The "issue" and "pull request" links get a 404:
https://github.com/beakerbrowser/unwalled.garden/blob/develop/issues
https://github.com/beakerbrowser/unwalled.garden/blob/develop/pulls

P.

Encryption and public keys

Hi Paul _o/

Great feature, love it! :)

My first thought when reading about the default schema was that this would be perfect to distribute public keys, somewhat like a decentralized keybase.

The main interest of the dweb for me is to not have my data owned by centralized third parties ; but if it's all public, it kind of misses the point. I guess I'm not the only one to investigate the dweb for privacy reasons? That's why I started to work with dat/beaker rather than ssb (as much as I love ssb), I love how it's easy with beaker to build apps encrypting the data and just sharing the encrypted files.

Right now, I do that in beaker by storing the private (ed25519) key in localStorage and telling the user to download it as a file, then either find it again from localStorage or retrieving the file through an input[type=file]. To share their public key with their friends, they're on their own, though (meaning, they'll use centralized solutions like facebook or slack to do it). Love how unwalled.garden would make that easy!

Did you consider building encryption within unwalled.garden and beaker? Like, having beaker store our private key, maintaining a local registry of friends' public keys through the unwalled.garden files, then leveraging the visibility field to add a "protected" value, which would allow to encrypt the content for a set of given public keys.

In any case, with the custom schemas from unwalled.garden, it will make it easy to distribute public keys anyway, so thanks! :)

Title and structure of posts

I could just manually create a public dat compatible with unwalled.garden, and later when UnwalledGarden is launched, link it with a private counterpart?

What is the direct script (that replies a output) used to name posts when they're posted? And their inner data scheme?

Why activity logs as "posts"?

Activity logs supporting lots of characters and being treated as posts? It is confusing.

And why a scheme for blogpost? Any post is a blogpost, as on DTube/Steemit. For microblogging with limited characters, this kind of post should support external schemas (peeps, tweets).

Questions about the intro

There were a few things I didn’t fully understand. I’m mentioning them here, in case you are interested in feedback (others may have questions similar to mine):

• Why are schema IDs stored in metadata fields? Why not inline in the files? Why do directories have a schemas?
• Do only my followers see my comments to a post?
• How do the predefined paths work – don’t they limit the extensibility? Why are both schemas and predefined paths needed?

Question about the File structure

Can i have /.data/my-app instead of /.data/unwalled.garden for data directory?

"via" post field

For example, showing in post: "via Fritter" or "via Beaker.Social for Windows"

[Question] Only Beaker, or a app that uses dat-node can work with?

Type definition for podcast episode?

Hi all, I'm curious if anyone created a type definition for a podcast episode with an array of tags. I have a need to organize a bunch of audio files on archive.org for a simple display on a wordpress page with filtering by tags and sorting by date of recording descending. I'm thinking that UG would help with this and can be used to generate multiple RSS feeds from the same material by filtering on the tags.

Switch between multiple identities with multiple `Person` schema files?

Can we switch between multiple accounts by having multiple person schema files?

API: protect a Dat, so any site that writes to it, can do only one commit at a time

[Discussion] UG on Chats

How UG standard would work on chats?
User would send its own messages on unique profile?

Migrating users from centralized platform to unwalled.garden

Hi everyone,

Thank you Paul for this exciting protocol and the excellent work on Beaker Browser! The 1.0 Beta version is amazing 😌

I'm a developer working on matters.news, a publication platform focusing on sane public discussions. It currently has around 40k register users and around 6m monthly visitor page views, mostly Chinese speakers around the globe.

We've been testing ways to bridge our users to p2p network, since we believe it's the only way moving forward as censorship intensifies and internet fragmentation continues (our site is already blocked in mainland China). We had a hard time choosing between Dat and IPFS, but decided to go with IPFS around 2 years ago. Since then we've integrated IPFS in our platform, utilizing public nodes to deliver contents, and prototyped a desktop client for publishing content without a server. But IPFS is still unstable albeit all the hype, and dynamic data with IPNS is far from production ready. Seeing the maturity of Beaker Browser and the proposal of unwalled.garden, we need to re-evaluate our decision.

Our original idea of the desktop application is essentially a WYSIWYG editor plus a RSS feed renderer, with some social features: an user can edit and publish content to her/his feed, subscribe to others' feed, and comment on others' content, which also gets published to their feed. We also need to bridge publications between matters.news and desktop client, so that the p2p network can have an attractive content base and user base to get through cold start.

From what I understand so far, the p2p RSS feed part is exactly what unwalled.garden is designed for. Instead of developing another desktop client, it looks like we can simply develop an unwalled.garden compatible web app, and direct our user to download Beaker Browser, enter the hash, while running a Dat daemon on our backend for migrating contents. But I still have several questions:

• What will the RSS/unwalled.garden support of Beaker Browser look like? For example, will it render RSS feed, allow user to subscribe, even allow user to comment? Or should them be implemented in a web app?
• Beaker Browser automatically setup a user's personal website compatible with unwalled.garden. If we want to implement more features, e.g. rich media WYSIWYG editor with image and video support, or BitCoin address for donation, is there a way to integrate into this personal website, or should it only exists inside a web app?
• One problem we need to solve is bridging a centralized server with p2p client for data persistency and discovery, similar to what SSB does with Pubs. Is this a general case that fits into the design of Dat/unwalled.garden?

I'm still exploring all the amazing works done in Dat/Beaker Browser space, so please excuse me if my questions are not yet well informed 😄

A webcounter

The first thing I did after making my first website in the 90s was add a web counter/hit counter. Checking back to see a higher number of visits made the experience sticky, and seeing larger numbers encouraged me to make my site bigger and better. It was miles before social media, but already didn't feel like talking to an empty room.

Figuring out how a webcounter might work on the distributed web is an interesting thought experiment. Following the design of unwalled.garden, here's how one might look and act:

000000789 visitors counted [Count me!]
__user1__, __user2__, __user3__ recently visited

Similar to comments, the count and user list are a result of the web crawler finding the appropriate record in the friend swarm of Dats.

"omni" post array

It points in which other networks the same post is baked, as example:

"omni":[{"url":"dat://6e96b27838c76991ac3563f729beb7c6e6c362fe369793784cd1c729b2b37299","net":"Namecoin"},{"url":"dat://6348fac3c70e916885ead7dce6927f01e02c0efd1a9273849891c23b7e7bac4d","net":"Peepeth"},{"url":"dat://c6e05a8a2924b148e60c6b2aab4d8d7fe3a923a14a13ab22119e8d13afd2baf7","net":"ZeroMe"},{"url":"dat://71b4951cd41b604bff8b6ceb6841d0364b7342afe6fbf0319627c84dc1067015","net":"Mastodon"}]

Already used in some posts on Fritter.

Activitypub compatibility?

How much activitypub compatibility is planned? It's basically the same thing: jsons are downloaded / uploaded / created to power a social network.

See:

• Mastodon (tweetdeck clone)
• Pleroma (twitter clone)
• Pixelfed (instagram clone)
• Peertube (youtube clone)
• etc etc etc....

Person schema and Profile API

Does it make sense on writing a ton of data into the person schema ? (or it must contain just the type, description and title fields?)
Also how can I add data to the profile (dat.json), its a simple write to a json file or it ll be like the POSTS API (await posts.add({body, visibility})) ?

RSS compatibility

Someone will be much more likely to move their blog to UG if RSS works with it. For example, it's not much of an effort to ask readers to switch their RSS URLs. Readers still get linked to your content via their feed readers etc.

Both RSS and ActivityPub are not built with resiliency in mind, so dat is a perfect compliment to it.

Similar to #19 with ActivityPub, I suppose the solution would be to build a service that converts the schemas.

It would be more adoption friendly to do this automatically upon post creation, but perhaps this would be a service more in the domain of the dat service that converts your dat site to an https site.

Name change from "post" to "status" and the purpose of Unwalled Garden?

I notice the name change from the more generic "post" to the more twitter-like "status".

Do you intend to make UG primarily for shorter posts that link out to other places / networks?

Since DAT is made for sharing files, wouldn't it be trivial to "post" video files or images or long blogs in unwalled garden like youtube or instagram or wordpress?

Seems like the name change indicates that a typical user is meant to have an UG profile as well as a DAT website where they host their primary material like videos, images, long blogs, etc. Is that right?

Might be worthwhile to create a couple of examples of how users are meant to use the network :)