GithubHelp home page GithubHelp logo

status-im / specs Goto Github PK

View Code? Open in Web Editor NEW
14.0 54.0 14.0 2.04 MB

Specifications for Status clients.

Home Page: https://specs.status.im/

License: MIT License

HTML 77.04% Ruby 2.05% JavaScript 20.91%
website

specs's Introduction

layout permalink nav_exclude
default
/
true

Specifications for Status clients

CI

This repository contains a list of specifications for implementing Status and its various capabilities.

How to contribute

You can read about how to build this project here.

  1. Create an issue for a new Status Improvement Proposal (SIP) or some bug that you'd like to address
  2. Create a corresponding PR and ping some existing SIP editors for review

If you need help, ask in #protocol at Status / Discord.

Specification style guidelines

Become familiar with the specification style guidelines to understand how you should write or amend specifications.

Spec lifecycle

Every spec has its own lifecycle that shows its maturity. We indicate this in a similar fashion to COSS Lifecycle:

At present (March 30, 2020) this means stable specs are what is in v1 of the Status App. Drafts and raw are work in progress specs.

Status Improvement Proposals (SIPs)

The main specification for writing a Status client is 1/CLIENT.

For all full index of all specs, see specs.status.im, especially stable specs.

Protocol Research

These are protocols that are currently being researched. These are designed to be useful outside of Status as well. To the extent that these protocols are used within Status clients, they will show up as SIPs in the future.

To see more on this, please visit the current home: vac protocol.

Continuous Integration

The site is built in Our Jenkins CI based off of master branch.

specs's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

specs's Issues

The road to V1 specs and more

Alright, this issue is a to track and describe what needs to be done to get to the start of V1 specs. For V1, I'm ok with the first non-draft version having overlap with specs that live in vacp2p specs. As things finalize there, we can remove them here and appropriately reference them here with our design decisions around that reference.

I'd also like to store various higher-level views of the Status App here which give insight to how our implementation of Status uses the specs within here. I'm open to how we document and store that, but initial attempts to do so can be found here:

Checklist of getting the bare minimums for V1:

Note that this does not HAVE to be done before V1 launch, but with the extra time between finishing required issues and launch date, it should give some time on documenting how V1 works.

Hit me with updates, changes, and thoughts.

Create a bandwidth-optimized node type proof-of-concept spec

Create a PoC node for trading off bandwidth restricted devices that cares less about metadata protection.

Rationale:

  • A large subset of users care a lot more about bandwidth usage than metadata protection
    • to the extent that few core contributors use Status on a daily basis
  • privacy guarantees of Whisper are spotty at best
  • a major overhaul of this layer is likely needed, but we can also work with what we got to get meaningful change for specific nodes on the order of ~weeks

Constraints:

  • not impact rest of network
  • not changing the current Whisper API in any major way (that impacts network)
  • premium on bandwidth

Trade-offs:

  • willing to sacrifice metadata protection
  • possibly lessened connectivity

Essentially Infura for chat.

Preview navigation children in left bar

Problem

Currently you have to be on a child node in ToC to see siblings:

image

These child nodes should be visible from main page, but currently they are not:

image

Notes

Probably a quick theme fix / option with https://github.com/pmarsceill/just-the-docs

I had some issues getting the whole Jekyll shebang running in a timebox, so just doing live edits right now. This change would benefit from tighter feedback loop by running locally with SASS or whatever setup is needed.

Decide on SIP process and format

What do we want this repo to look like in terms of format of separate followed specs?

Are we to follow the given example of bittorrent? If so, then we need to come up with a process of BEP/EIP/BIP-like process.

I suggest we follow the following RFC2119 with regards to wording.

Replace forked Whisper usage with Waku/0 in specs

Problem

Currently we use a forked version of Whisper that isn't documented in any great detail. This makes it difficult for other clients to implement and get compatibility with Status clients, e.g. status-nim. It also creates confusion around what requirements and guarantees we have, since we are de facto doing more than EIP627 and this has implications around UX, privacy, DDoS resistance, etc (mailservers, light node, rate limiting).

Acceptance criteria

  • Status Client spec reference Waku spec
  • Updated Waku mode usage spec (replace Whisper Usage and Mailserver)

As well as issues/dependencies marked out for for :

  • Current specs for v1 marked clearly as such
  • Deprecation/recommendation notice for Waku mode for client implementers (e.g. status-nim)

Details

See vacp2p/rfc#27 for more details.

This issue is largely dependent on above, but some work can be done in parallel to ease that work. I.e. by documenting customization to Whisper in our current specs, such as:

  • mailserver usage
  • light mode
  • extra packet codes
  • rate limiting
  • any other changes we have done to EIP627 that aren't accurately captured in our specs (litmus test: 100% compatibility with status-nim w/o reading status-go/whisper code)

Possible solutions

See acceptance criteria.

Notes

Add new methods supporting chat functionality to status api in DApp browser

Some features that have been requested in the past and would enhance DApps in a unique way that only Status can provide is exposing the chat api to DApps.

Methods include being able to send messages on behalf of a user and reading conversations.

Because of the security and usability implications it would make sense to draft a specification first.

examples:

window.ethereum.status
.sendToPublicChat('room-name', 'message')
.then(data => {
console.log('message:', data.message)
})
.catch(err => {
// On user rejecting request or other reason
console.log('Error:', err)
})
window.ethereum.status
.getPublicChatMessages('room-name')
.then(data => {
// type messages :: [ { name, pubKey, message, timeStamp } ]
const { messages } = data
console.log('messages:', messages)
})
.catch(err => {
// On user rejecting request or other reason
console.log('Error:', err)
})

Send images

Image Sending Specs

cc @errorists @flexsurfer :)

User Stories

As a user, I want to send media from my device in 1:1, group and public chats.

  • I want to take new photos/videos and send them.
  • I want to upload existing photos/videos from my device and send them.

POC: status-im/status-mobile#9455

  • Relies on IPFS rather than Status IPFS service.
  • Reduce image resolution when uploaded.
  • Camera flow is not complete yet.
  • Users can not enlarge images in chat.
  • Image aspect ratio not maintained in thumbnail.

MVP: Still images only

  • iOS and Android
  • 1:1, group and public chats
  • Still images only
  • As a sending user, I can:
    • Launch camera and take a new image from chat
    • Upload an existing image from my device
    • Note: There is a wait while the image uploads; it appears to be okay if the user navigates away from the chat (e.g. to Stickers tab) while uploading, but should be tested further.
    • Send image OR text; can not couple them in the MVP
  • As a receiving user, I see image thumbnails in their proper aspect ratio within a chat.
    • I can tap on an image for a full screen preview.

Technical Requirements

  • Images will be pinned using Status IPFS service.
  • Images will be read from Cloudflare.
  • Need to decide which resolution to store in order to display different sizes of the image.
    • Possibly upload two versions: one for the thumbnail, and one for the full size view.
    • We'd have to upload the image twice and it would have two different hashes. You would click the thumbnail, see a loader, and then see the full size image.
    • Even small file sizes on our gateway can take 1 minute to upload.
    • If we have 1,000 people uploading images, it could be a problem. Should discuss with Jakub.
  • IPFS hash should be changed to multihash (similar to stickers).

v1, v2: UI enhancements

To be prioritized according to user feedback. Ranked here according to effort/complexity:

Technical Requirements

  • Some of these ideas require interaction with native parts, e.g. downloading and copying, and thus can be quite tricky.

Future versions: Send video and other files

  • Requires more supportive protocol than IPFS

Initial Message Payload Specification

This should give an overview of our message payload and contents. What's special about this one is that it doesn't really provide any security guarantees (or at least not many).

Things like upgradability are a concern here, as well as how to deal with new types (e..g what is a sticker in a CLI client?).

One of thinking of these is more like different clients, so TTT and Sticker market and Extensions would be their own specification. For now we can keep them listed in this initial spec, especially MUST ones like text/plain.

Also considerations re Transit, e.g. if we want to move to a different format like Protobuf. How would compatibility work.

Also considerations re what happens if someone puts a bad timestamp or clock.

See x9.md for current draft.

Acceptance criteria

  • Someone can read this and implement a Status client (including links to other specs)
  • They should clarify what security guarantees the protocol(s) provide and don't provide
  • As well as what each protocol/layer requires and provides
  • They should be described as orthogonal pieces, so if someone wants a different transport that should ideally require minimal tweaks to the protocols

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Personal nodes

  1. Research
  2. Decide on viability
  3. If viable, write specs
  4. Implement

status-im/swarms#28

Goals:

at first a private node would allow a user to reduce bandwidth consumption (from messages and wallet) to a minimum on his phone
for iOS it means he could also set up notifications.
that would mean a much lightweight version of status running on the phone, and much more independence from status infra for little cost
later on there is other kind of nodes that would be developed which would support public requests such a wallet history or ipfs/swarm content.
nodes could be paid for services/responses using the raiden network, snt transfers would be almost instant and free which would also allow for features like snt reactions and tribute to talk to exist

Transaction history management

Specification should include:

  • how tx history is requested from Infura
  • functions included in Status-go
  • how and when status-react requests tx updates for outgoing and incoming transactions
  • how status-react handles pending transactions
  • make clear what changes are required when moving away from Infura (different endpoint for archival data, rewriting recent txs to limit dependence on archival data)

Implementation matrix for Whisper/Waku clients

Problem

What it means to support Whisper/Waku isn't clear, as there are many different components of it. Geth/nimbus/parity should be added to spec of where lightnode implementation exists for waku

Details

Dimensions to capture:

  • light mode
  • mailserver server
  • mailserver client
  • shh/6
  • waku/0

Code bases (with relevant version number / commit hash of need be):

  • geth
  • status whisper
  • parity
  • nim-eth
  • javascript? (Embark)

Acceptance criteria

  • Matrix with implementation Status of all dimensions and implementations
  • It can be a WIP, so acceptance criteria OK to have empty rows, etc.

Move from transit to protobuf

We decided to move from transit to protobuf.

The main reasons for this are:

  • Consistency. We use protobuf already for most of the messages, and transit only for the application layer.
  • Better interop with other languages. Protobuf has better support in various languages then transit.
  • Security. Protobuf has a set of well maintained libraries, which will avoid having to use less tested/popular libraries, which might not have gone through much scrutiny.

The proposed schema for protobuf is here status-im/status-protocol-go#128 , please comment there.

Identifiers

Might go together with URI scheme if it makes more sense to document how identifiers are used in the scheme

Conforms DID specification requirements by W3C where possible. This may include, and is not limited to the information below. Up to your judgement and extend of implementation.

For all identifiers

  • DID syntax (name, characters, id)
  • Ownership
  • Proof
  • Delegation (if relevant)
  • Discoverability/Methods/Queries
  • Interoperability
  • Portability
  • Authentication
  • Controller
  • Contact for change requests

References:
Example: https://handbook.scuttlebutt.nz/concepts/identity
https://w3c-ccg.github.io/did-method-registry/#the-registry

URI scheme

Might be paired with an identifier spec if it makes more sense to document how they are used in the URI scheme

Structure of this spec should include:

  • Scheme syntax
  • Scheme semantics
  • Encoding considerations
  • Security considerations
  • Applications that use the scheme (including e.g. join.status.im, dapps, Keycard)
  • Operations
  • Context of use
  • Contact for change requests

For reference see
https://tools.ietf.org/html/rfc7595
https://en.wikipedia.org/wiki/Magnet_URI_scheme
https://handbook.scuttlebutt.nz/concepts/identity

Content should cover

  • Identities
  • Transactions (any EIP's)

Initial Conversational Security Specification

This should give an overview of how we provide conversational security in Status.

See https://github.com/status-im/specs/blob/master/x6.md for current draft (kudos to @cammellos @pombeirp), as well Adam's initial spec.

The main issue with the current PFS whitepaper is that it treats PFS as a special thing, as opposed to talking about conversational security more generally (this might be more of a naming thing though - in any case it shows that we think of PFS as something (too) special). It's also not clear enough in terms of what guarantees we always make. We might also want to mention aspects like PCS.

As well as general evaluation based on SoK Secure Messaging, see inline doc.

Acceptance criteria

  • Someone can read this and implement a Status client (including links to other specs)
  • They should clarify what security guarantees the protocol(s) provide and don't provide
  • As well as what each protocol/layer requires and provides
  • They should be described as orthogonal pieces, so if someone wants a different transport that should ideally require minimal tweaks to the protocols

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Questions spec should answer

Security and Privacy

Confidentiality
Integrity
Authentication
Participant Consistency
Destination Validation
Forward Secrecy
Backward Secrecy
Anonymity Preserving
Speaker Consistency
Causality Preserving
Global Transcript
Message Unlinkability
Message Repudiation
Particip. Repudiation
Adoption

Out-of-Order Resilient
Dropped Message Resilient
Asynchronicity
Multi-Device Support
No Additional Service
Group chat

Computational Equality
Trust Equality
Subgroup Messaging
Contractable
Expandable

Example technologies

Trusted servers (mailservers?)
Double ratchet
X3DH
Prekeys

Also note that multidevice fits here, fyi @decanus

Document Lamport unix time bidding process

Currently https://github.com/status-im/specs/blob/978c5b77bb45805df91a326132b4bc89671006d2/status-payloads-spec.md#clock-vs-timestamp-and-message-ordering mentions use of Lamport clocks.

However, it doesn't appear to mention the unix time bidding process tweak and rationale (with Lamport invariant) described in https://github.com/status-im/status-react/blob/c9994b5d0f72377ec51000541e6b02500f8430f5/src/status_im/utils/clocks.cljs

;; But what we can do, is to use our time to make a "bid", hoping that it will
;; beat the current chat-timestamp. So our Lamport timestamp format is:
;; {unix-timestamp-ms}{2-digits-post-id}

cc @corpetty @cammellos

Change 8-eips.md to 8-supported-specs.md

Introduction

8-eips.md currently references both BIPs and EIPs. Granted EIPs are the majority of the specs referenced. However with the inclusion of Vac specs in to this document we would be adding yet another source of specification to the 8-eips.md document.

Proposal

  • Add Vac specs into 8-eips.md
  • Change name of 8-eips.md to 8-supported-specs.md

Question

  • Do we want to add Vac specs to 8-eips.md?
  • Do we want to change the name of 8-eips.md to 8-supported-specs.md?

Status protocol stack specifications, umbrella issue

Umbrella issue to capture documenting current protocol specification effort. Should pass the following litmus tests:

  • Someone can read these set of specifications and implement a Status client
  • They should clarify what security guarantees the protocol(s) provide and don't provide
  • As well as what each protocol/layer requires and provides
  • They should be described as orthogonal pieces, so if someone wants a different transport that should ideally require minimal tweaks to the protocols

Additionally, we should have a process in place that's:

  • clear in terms of format and and process (see #1)

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Getting validation and evaluation outside of Status is desirable, but out of scope of this issue.

I cut up the amazing work @adambabik did into a few pieces as an example of how I think we should look at it. This is also how other protocols and papers roughly structure things, as far as I can tell. Each of the items above applies to each of these.

  • x4 | Initial Protocol Overview (#11)
  • x5 | Initial Trust Establishment Specification (#12)
  • x6 | Initial Conversational Security Specification (#13)
  • x7 | Initial Transport Privacy through Whisper Specification (#14)
  • x8 | Initial Message Payload Specification Specification (#15)

Finishing these pieces amounts to getting the first set of SIPs into an accepted state.

Removal of Personal Pronouns From Specs

Introduction

The specifications have a mix of formal and informal language. Use of personal pronouns is generally common in informal text, and generally avoided in formal text.

It has been identified that:

Example

The other node types are optional, but we RECOMMEND you implement a Mailserver

The other node types are optional, but it is RECOMMEND to implement a Mailserver

Proposal

  • Decide if specifications should have a consistent tone and mode of language (Answer: Yes)
  • If a consistent tone is required, what the tone should be (Formal tone)
  • Implement the changes in language

Notes

These changes could also have a major impact on the structure of the specification documents. For example:

1-client.md#p2p-overlay-1 has the following subsections

  • Why devp2p? Why not use libp2p?
  • What about other RLPx subprotocols like LES, and Swarm?
  • Why do you use Whisper?
  • I heard you were moving away from Whisper?
  • Why do you not use Discovery v5 for node discovery?
  • I heard something about mailservers being trusted somehow?

These sections are more approachable and easier to read as they are written in an inviting and engaging tone. I feel that we should make a distinction between what is supplementary information that can be written in an informal manner and what is instructive or descriptive information that is better suited to formal language.

References

Describe the rate limiting feedback for peers

Problem

Due to using a low PoW, Status nodes and clients are vulnerable to a denial of service attack.

It's not possible to increase PoW because of mobile usage. The alternative is an implementation of rate limiting.

Solution

Describe in the spec, how rate limiting would work. It should take into consideration:

  • the general algorithm
  • rate limiting settings on the client and server
  • how rate limiting information is broadcasted to peers (similar to broadcasting PoW and bloom filters settings)
  • rate limiting information in the handshake.

Initial Transport Privacy through Whisper Specification

This should give an overview of the transport privacy layer through Whisper. It should also mention requirements such as Devp2p (plus possible workarounds like libp2p mirror (side note)).

It should be structured in such a way that Whisper is an implementation detail, and it can be replaced by another spec, such as use of PSS, or Tor, or local network, or mixnet, etc.

We have a great start with @adambabik's document.

See https://github.com/status-im/specs/blob/master/x8.md for current draft.

Acceptance criteria

  • Someone can read this and implement a Status client (including links to other specs)
  • They should clarify what security guarantees the protocol(s) provide and don't provide
  • As well as what each protocol/layer requires and provides
  • They should be described as orthogonal pieces, so if someone wants a different transport that should ideally require minimal tweaks to the protocols

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Questions spec should answer

Properties such as:

Privacy

Sender Anonymity
Recipient Anonymity
Particip. Anonymity
Unlinkability
Global Adv. Resistant
Usability

Contact Discovery
No Message Delays
No Message Drops
Easy Initialization
No Fees Required
Adoption

Topology Independent
No Additional Service
Spam/Flood Resistant
Low Storage
Low Bandwidth
Low Computation
Asynchronous
Scalable

Example technologies

Store-and-forward for mailserver
Messaging broadcast for Whisper
Specifically mention blockchain and how messages are not on blockchain but uses same devp2p stack

Update spec with mailserver's p2pRequestCompleteCode packet

Problem

p2pRequestCompleteCode packet is used by mailservers to tell a requester that all envelopes were forwarded. It also contains some metadata like cursor.

Solution

Add to the spec how the payload of p2pRequestCompleteCode looks like.

Mentions in chat

Mentions Specs v2

User Stories

As a (sending) user, I want to get the attention of certain individuals within group or public chats.

As a (receiving) user, I want to notice when someone calls me out or responds to me directly in a group or public chat.

Iterations

MVP: Discord style mentions: Visual distinction only.

v1: Mentions & replies via PN for Android: Visual distinction, scroll-to-position when opening from PN. <— Heaviest/most variable and research-intensive scope of work.

v2: In-app notification center for mentions & replies, scroll-to-position when tapped.

MVP: Discord style mentions: Visual distinction only


(Design WIP)

  • As a (sending) user, I type the @ symbol to pull up the list of users in a chat—both random names and ENS—and either type or select the person I want to mention.
    • Acceptance criteria:
      • Typing @ produces a list of names
      • Both ENS and random names
      • User can type the name or select it from the list; as they type the selection of names is narrowed.
  • As a (receiving) user, my mentions appear highlighted within a chat.
    • Acceptance criteria:
      • Visual distinction for each mention.
      • Visual distinction between ENS & non-ENS names.

Work required

  • Go/protocol: Bulk of the work for MVP. Status-go will parse messages to check for mentions. Mentions will be based on a user's pubkey.
  • React: UI is very similar to previous text commands, which can be re-used. We will experiment checking a user's list of known users in react and including ENS name if relevant. If too expensive, we will move to status-go. A few days of work.

Questions & Constraints

  • When does the visual distinction change/fade? e.g. how is a mention marked as read?
  • Concern that receiving mentions in status-react will become too expensive if mentions are frequently used.

v1: Add push notifications: Visual distinction + PN on Android + scroll-to-position


(Design WIP)

  • As an Android user, I receive a push notification when somebody mentions or replies to me in a group or public chat.
    • 3 possible variations depending on system settings:
      • Preview available/mention
      • Preview available/reply
      • Previews hidden
  • When I open the PN, the chat scrolls to the position of the mention.

Work required

  • Go/protocol: Parsing as much as possible in status-go.
  • React: Research required. Likely we will want to pull the mentioned message + 10 on either side of it for buffer. Additional messages load as user scrolls.

Questions & Constraints

  • Can we make scroll to position work for this feature?
    • Main challenge is what to do with all the messages that need to be loaded in between the user's last loaded message, and the mention (if it's 1000 messages back, for instance, that's a performance issue)
  • It is not worth implementing scroll-to-position for iOS because the complexity is too great when dealing with multiple mentions. The better solution is to build v2 - in-app notification center.

v2: In-app notification center: Visual distinction + PN on Android + in-app notification center + scroll-to-position


(Design WIP)

  • As a user, I can visit my notification or activity center at any time to see a list of my recent mentions and replies.
  • When I open a mention or reply from the notification center, I scroll to the position of the message within chat.
  • When I receive a new mention or reply while using Status, it appears in a banner in real time at the top of my screen.

Questions & Constraints

  • How long do notifications live in the notification center?
  • Will not see historical mentions after you first install the in-app notification center build.

Work required

  • Go/protocol: As mentions are parsed out, create a different db entry. To show a notification, create another entry.
  • React: UI is a few days of work.

Future versions

  • Invite people outside of a chat into chat
    • contacts or non-contacts

Fix Whisper usage spec wrt mailserver

As a client implementer, I want specs to describe how to use mailservers, not how the Status app works.

Right now

Currently this spec: https://github.com/status-im/specs/blob/master/status-whisper-usage-spec.md#request-historic-messages explains the boring / non protocoly part: status-react to status go.

Expected

Instead, it should explain the interface between status-go and mailservers. Specifically:

"The above topics is then converted into a bloom filter and then and sent to the Mailserver." should be expanded on, and the "Object - The message request object: [...]" part from status-react should probably be removed.

^ @adambabik @corpetty

Initial Trust Establishment Specification

Trust establishment refers to the combination of long-term key exchange and long term key authentication.

This should answer how we establish trust in Status, i.e. between clients, and any other type of trust establishment (like mailservers etc).

See https://github.com/status-im/specs/blob/master/x5.md for current draft.

  • If certain things are deemed as out of scope, e.g. ENS usernames, LES, etc then this MAY be excluded from an initial version, assuming it is documented as an Enhancement in the document.

Acceptance criteria

  • Someone can read this and implement a Status client (including links to other specs)
  • They should clarify what security guarantees the protocol(s) provide and don't provide
  • As well as what each protocol/layer requires and provides
  • They should be described as orthogonal pieces, so if someone wants a different transport that should ideally require minimal tweaks to the protocols

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Questions a spec should answer

Security properties

Network MitM Prevented
Operator MitM Prevented
Operator MitM Detected
Operator Accountability
Key Revocation Possible
Privacy Preserving
Usability

Automatic Key Initialization
Low Key Maintenance
Easy Key Discovery
Easy Key Recovery
In-Band
No Shared Secrets
Alert-less Key Renewal
Immediate Enrollment
Inattentive User Resistant
Adoptability

Multiple Key Support
No Service Provider
No Auditing Required
No Name Squatting
Asynchronous
Scalable

Examples

  • Opportunistic Encryption
  • Key Fingerprint Verification
  • Key directory
  • Web of trust
  • Optional verification
  • Blockchain

Relevance

  • Conversational keys
  • Identity of mailserver
  • ENS
  • (Later) Blockchain relevance
  • (Later) Multiple key support

User referral program

Referral Implementation Steps

Source

Android Only

Goal

We need to acquire users.

We will do so allowing people to append an invite code to a join.status.im url, in which they'll earn SNT for successful onboarding and/or purchase or starterpack

Practically speaking this will look like:

  • Friend Referrals (Earn SNT for referring friend)
  • Advertising Campaigns promoting DApps

This is the first non-idealistic iteration of our user aquisition engine as promised in the whitepaper.

App Modifications

Android Only

  • App Install Attribution for both Google Play and Apple App Store
    • Android com.android.vending.INSTALL_REFERRER
    • iOS Advertising Identifier (IDFA/SKAdNetwork)
  • out exists in referral data, treat as universal link and open chat/public chat/dapp browser after onboarding
  • If a invite code exists in referral data, make an HTTPS post Request after Onboarding
  • Android Change Image to show "Earn SNT, Invite a Friend" (Ask Carl for copy)
  • include invite code with starterpack receipt validation

Attribution & Payout Service

Jarrad will do this, only focus on the app modifications.

Flows

Friend shares a join link with an invite code (join.status.im/0x0...?invite=f79989f679386be7aa56dc1c4) to new user
new user opens link (we track them)
link redirects new user to google play / app store / get page
depending on the out variable in referrer the app should after onboarding:

  • open chat with user
  • open chat with public chat
  • open webpage in dapp browser

similar to how a universal link works if the app is installed

there are 2 postbacks to join.status.im with the invite code

  • after onboarding join.status.im/event
  • when validating starterpack purchase join.status.im/event

Implementation

Android

This is only for builds deployed to Google Play only, this won't work for direct apk installs and the code shouldn't be present.
(not independent apk)

  • User presses Invite friend or share their chat key (public key)
  • App makes a http post request to https://join.status.im/register
    • returns success or failure reason
      the invite code is a truncated hash of the signature, which is constructed by producing a json as a msg, signing the msg, and producing a json that contains the signature information:

The actual payloads for registration and events look like this
https://github.com/status-im/starterpack-service/blob/master/scripts/geth.js

and will record a the invite code, a truncated (first 25 chars) sha3 of the signature.
the app can generate invite codes itself for instant referral links

web3.sha3(sig).substring(2,27)
'f79989f679386be7aa56dc1c4'
  • the app generates link and the user shares it with their friend

https://join.status.im/0x04abcXYZ123...?invite=f79989f679386be7aa56dc1c4
https://join.status.im/@jakubgs?invite=f79989f679386be7aa56dc1c4
https://join.status.im/@jakubgs.eth?invite=f79989f679386be7aa56dc1c4
https://join.status.im/markets?invite=f79989f679386be7aa56dc1c4
https://join.status.im/b/ens.domains?invite=f79989f679386be7aa56dc1c4

which will redirect to

https://play.google.com/store/apps/details?id=im.status.ethereum&referrer=out%3D0x04abcXYZ123...%26invite%3Df79989f679386be7aa56dc1c4
https://play.google.com/store/apps/details?id=im.status.ethereum&referrer=out%3D@jakubgs%26invite%3Df79989f679386be7aa56dc1c4
https://play.google.com/store/apps/details?id=im.status.ethereum&referrer=out%3Db%2Fens.domains%26invite%3Df79989f679386be7aa56dc1c4

additionally advertisers require 1 more parameter cid so join.status.im will also have to forward this param to gplay
ie
https://join.status.im/0x000...?invite=f79989f679386be7aa56dc1c4&cid=foo
https://play.google.com/store/apps/details?id=im.status.ethereum&referrer=out%3D0x000...%26invite%3Df79989f679386be7aa56dc1c4%26cid%3Dfoo

out is the post-onboarding action to perform (same as universal link)
invite is the invite code (needs to be sent in /event postback and /validate)
cid is clickid for advertising networks (needs to be sent in /event postback /validate)

WARN: this is pseudo-code
better to use the install referrer api below

We need to create a BroadcastReceiver that can record the values

and register it in the AndroidManifest.xml

<receiver
android:exported="true"
android:name="im.status.ethereum.receivers.InstallReceiver">
<intent-filter>
<action android:name="com.android.vending.INSTALL_REFERRER" />
</intent-filter>
</receiver>
@Override
public void onReceive(Context context, Intent intent) {
        String rawReferrerString = intent.getStringExtra("referrer");
        if(rawReferrerString != null) {
// do stuff
Map<String, String> referralParams = new HashMap<String, String>();
try {
referrer = URLDecoder.decode(referrer, "x-www-form-urlencoded");
} catch (UnsupportedEncodingException e) { return; }

// Parse the query string, extracting the relevant data
String[] params = referrer.split("&");
for (String param : params) {
String[] pair = param.split("=");
referralParams.put(pair[0], pair[1]);
}
}
}

Once we receive the parameters

Apple

Android Only

Spoke with Carl, you're right, let's forget Apple.

Without our own Advertiser Keys or Skadnetwork - iOS doesn't seem feasible.

I have no idea how IDFA works but it can be used for app install attribution
Looks like apple rolled out skadnetwork in StoreKit
https://developer.apple.com/documentation/storekit/skadnetwork
https://developer.apple.com/documentation/storekit/skadnetwork/verifying_an_install_validation_postback

Android Google Play Install Referrer

Play Store will no longer broadcast the "install_referrer" in March 2020, this is a new way how it should be used

https://developer.android.com/google/play/installreferrer

Google Play Install Referrer

You can use the Google Play Store's Install Referrer API to securely retrieve referral content from Google Play, such as:

The referrer URL of the installed package.
The timestamp, in seconds, of when a referrer click happened.
The timestamp, in seconds, of when an installation began.
Whether the user has interacted with your app's instant experience in the past 7 days.

Requirements

The Install Referrer API is exposed by the Google Play Store app on a device. Devices with a Google Play app version of 8.3.73 or later automatically have access to the API.

You must also have a Google Play Console account to use the Install Referrer API.

ReferrerDetails response = referrerClient.getInstallReferrer();
String referrerUrl = response.getInstallReferrer();
long referrerClickTime = response.getReferrerClickTimestampSeconds();
long appInstallTime = response.getInstallBeginTimestampSeconds();
boolean instantExperienceLaunched = response.getGooglePlayInstantParam();

Caution: The install referrer information will be available for 90 days and won't change unless the application is reinstalled. To avoid unnecessary API calls in your app, you should invoke the API only once during the first execution after install.

RN lib: https://github.com/Gretchin/react-native-install-referrer

The big question is , how to test this API ?
Old way, can't be used now https://tune.docs.branch.io/sdk/testing-the-google-play-install-referrer/

Second question how we'll track reinstalls ? say user onboarded reinstalled app and onboard again
[JH] this is done server-side, not an app concern

status-react implementation steps

  1. add referrer dependency
  2. after fist onboarding getInstallReferrer and call https api?
  3. done ?

PR: status-im/status-mobile#10139

  • read referrer
  • call https with referrer

Document how message confirmations work

As a user,
I would like to know whether my message is properly sent through the wire and received by peers
so that I can be sure it has a chance to be propagated and received by others.

Acceptance criteria

  • Describe the current mechanism of message confirmations.
  • Document all packet codes used to achieve this.
  • Describe the role of peers in message confirmations.

Document added Whisper packet codes

We would like to document all Whisper packet codes that extend EIP-627 so that other implementation can be complete in terms of feature set.

Acceptance criteria

  1. Determine which packet codes present in our Whisper fork are actively used.
  2. Document used packet codes with their purpose.

References

Initial Protocol Overview

This should give an overview of the protocol stack. Design goals, roughly mention each layer, what this gives in terms of guarantees, as well as specific considerations when it comes to using the layers together.

Things like how we do transport should to the greatest extent possible be an implementation detail, and instead be explained in a different document.

See https://github.com/status-im/specs/blob/master/x4.md for current draft.

Acceptance criteria

  • One can judge whether this protocol meets their requirements (functional as well as security) and technical stack.
  • It should provide an intuition how the protocol works as a whole.
  • Layers should be described as orthogonal pieces and what interfaces (if any) are required between them.

In terms of who will judge, it'll be 2-3 main groups initially:

  • Ourselves in protocol group
  • Client implementers (Core, Embark and Nimbus people i.e. status-js and Stratus)
  • Core dev calls participants

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.