kodadot / loligo Goto Github PK
View Code? Open in Web Editor NEWCommon good repository for Squid indexers in KodaDot
Common good repository for Squid indexers in KodaDot
> 9th of April
Will take that time to design new Schema 2.0 ๐
Originally posted by @vikiival in kodadot/nft-gallery#5388 (comment)
Tasks:
displayPrice
May need help from @exezbcz for task 1.
type NFTEntity @entity {
buyCount: Int!
}
So anytime someone do buy just top up the value
We see a influx of porn ART on KodaDot
The task:
Introduce lewd
boolean type into Collection / NFTs
read metadata of collection/item (attributes) and set the lewd attr accordingly.
The egde case:
It could happen that collection is not marked as nsfw but in reallity is.
so we must update the collection to lewd = true
if at least one NFT is NSFW
Steps:
lewd
attribute for both collection and NFTlewd
tag like thisNote: when you are minting new nft, and nft is lewd, updtate also collection to lewd
To make some trading views easier the optional path is to make a Trade entity like following
type TradeEntity @entity {
id: ID!
nft: NFTEntity!
createdAt: DateTime!
closedAt: DateTime!
initialPrice: BigInt!
closingPrice: BigInt!
realized: BigInt!
percent: Int!
profit: Boolean!
issuer: String!
buyer: String!
}
May @daiagi can have more information since he was working on Flipper UI in Kodadot
With #5 we should have unified schema across Squids
What that mean?
What I am currently missing in BAO is ability to logically fetch metadata of NFTs and Collections created under specific time. because we are running cron this time should be matched;
SELECT DISTINCT metadata as id, image
FROM nft_entity ne
JOIN metadata_entity me on me.id = ne.meta_id
WHERE ne.created_at >= NOW() - INTERVAL '$1';
SELECT DISTINCT metadata as id, image
FROM collection_entity ce
JOIN metadata_entity me on me.id = ce.meta_id
WHERE ce.created_at >= NOW() - INTERVAL '$1';
https://www.postgresql.org/docs/current/functions-datetime.html
let's say default interval should be 15 minutes
INTERVAL '15 minutes';
The current listing in events is nice but we need to extend NFT entity with listing
type Listing @entity {
id: ID!
nft: NFTEntity!
collectionId: String!
displayPrice: BigInt!
royaltyAmount: BigInt!
total: BigInt!
updatedAt: DateTime!
createdAt: DateTime!
}
We can push metadata from squid directly to CF R2 via S3API
As we want to gradually get listed here on these charts, we would need some kind of endpoint so we can get integrated to various dashboards
I think we need to show there Volume 1d & 7d, trades in 1d & 7d, TVL would be also nice.
I would split it by protocol, chain, NFT standard
https://defillama.com/nfts/marketplaces
Magiceden was ranking other places too back then, can't find it now, but they have at least this
Something like
collectionMetaById($id: String) -> Metadata
nftMetaById($id: String) -> Metadata
Just a draft out of my head;
There should be a new Entity call Holding
that tells us the stats about how each user holded the NFT.
type Holding @entity {
id: ID!
issuer: String!
nft: NFTEntitity!
obtainPrice: BigInt
releasePrice: BigInt
releasedTo: String!
releasedVia: Interaction!
obtainedAt: DateTime!
releasedAt: DateTime!
holdTime: BigInt!
}
hold time Should be denominated in hours? or days?
ref:
Currently this config is available only for snek and I need to replicate it to:
So that means TokenEntity is expecting same image under one collection right?
correct, all of the nfts under one collection that share same ipfs URL are a single TokenEntity
Originally posted by @daiagi in kodadot/nft-gallery#7462 (comment)
Task:
TokenEntity
add CollectionEntity
type TokenEntity @entity {
collection: CollectionEntity
}
We are slowly getting to the tipping point where we need to replace user as String!
with UserEntity
Would would it do?:
REF:
to NFTEntity
add new field sequence
that should be either BigInt
or Int
type NFTEntity @entity {
sequence: BigInt! // or Int!
}
How to fill that?
Derive the number value from sn
sn = '00000000000008'
sequence = BigInt(sn)
Because sorting based on SN is not correct (because sn is String)
I am not sure how bigint would behave in sort (maybe check first some ordering via blockNumber if it's correctly sorted)
Yeah the other alternative is to remap sn into BigInt or Int
but I am not sure what will we break on frontend
Currently we are using a really ugly hack on frontend to determine if user can mint in the collection.
totalCount: ce.nfts?.filter((nft) => !nft.burned).length,
}))
.filter(
(ce: MintedCollectionKusama) =>
(ce.max || Infinity) - ce.alreadyMinted > 0
)
The task:
Implement a resolver that would take address as parameter and returns a list of collections that could be used for minting
The stuff should be that instead of saving price as 1e12 I would save it as 1
that means
const price = Float(BigInt(interaction.value || '0') / BigInt(`1e${decimals}`))
THE Task:
displayPrice: bigint!
Value of the displayPrice should be:
price + 3% of price (marketplace fee) + n% of royalty (if supported by chain)
I added the package
label because fee calculation should be done there ideally
How to calculate stuff can be found in nft-gallery
under name somePercentFromTx
.
It is possible that current on/off-chain attributes can be parsed not very well.
such as addresses
attributes look good on canary: https://canary.kodadot.xyz/stmn/gallery/34-2, except substrate addresses @vikiival I checked our code and it is most definately a string: "DhvRNnnsyykGpmaa9GMjK9H4DeeQojd5V5qCTWd1GoYwnTc"
Originally posted by @niklasp in Proof-Of-Chaos/website#68 (comment)
Thing is that RMRK2, Ajuna, Statemine has ability to add on-chain attributes.
type Property @jsonField {
key: String!
value: String!
}
What to do?
for collection fields add index to:
type CollectionEntity @entity {
createdAt: DateTime!
distribution: Int!
floor: BigInt!
updatedAt: DateTime!
nftCount: Int!
supply: Int!
volume: BigInt!
}
On NFT Entity
type NFTEntity @entity {
burned: Boolean!
createdAt: DateTime!
name: String @index
price: BigInt
updatedAt: DateTime!
}
Tasks:
Current series insight
type Series @entity {
id: ID!
issuer: String
unique: Int!
uniqueCollectors: Int!
sold: Int! @index
total: Int!
averagePrice: Float
floorPrice: BigInt
highestSale: BigInt
buys: Int
volume: BigInt
name: String! @index
metadata: String
image: String
emoteCount: Int
}
we can offload to like Insight
entity
blockNumber
add the block when the tokenEntity has been created
updatedAt
update the date when the tokenEntity's count (#50) or price (#52) has been changed
createdAt
add the date when the tokenEntity has been created
While reviewing kodadot/nft-gallery#5120 I found query that was added in 5f64a482a0e7cdcdb92341d314fdc89ba3eaf86e
query collectionStatsById($id: String!) {
stats: collectionEntityById(id: $id) {
id
base: nfts(where: { burned_eq: false }) {
id
currentOwner
issuer
}
listed: nfts(where: { price_gt: "0" }) {
id
price
}
sales: nfts(where: { events_some: { interaction_eq: BUY } }) {
id
events(where: { interaction_eq: BUY }) {
id,
meta
}
}
}
}
The reason is that when we returns this big chunks of data on frontend it creates more workload on frontend (to map it and parse)
To be able to show properties in sidebar we need to write a resolver.
resolver needs return list of attributes.
SELECT me.attributes from token_entity
left join metadata_entity me on token_entity.meta_id = me.id
where token_entity.collection_id = ${id};
Deliverables:
I want to see a resolver that takes collection id as parameter and returns a list of attributes
Ref:
Blocked by?
Do we still have access to infura_kodadot1
?
Task is to fetch mime-type in indexer for animation_url if present
import { obtainMimeType } from '@kodadot1/minipfs'
const type = await obtainMimeType(metadata.image)
const animationType = await obtainMimeType(metadata.animationUrl)
query hotNfts(
$orderBy: [EventOrderByInput!] = timestamp_DESC
$gte: DateTime!
) {
result: events(
where: { interaction_eq: BUY, timestamp_gte: $gte }
orderBy: $orderBy
) {
timestamp
interaction
nft {
collection {
name
id
}
}
meta
}
}
data we calculate
const collectionMap = groupBy(data, 'nft.collection.id')
const sortOrder = [...new Set(data.map((e) => e.nft.collection.id))]
const result = sortOrder.map((colId, idx) => {
const collection = collectionMap[colId][0].nft.collection
const nfts = sortBy(collectionMap[colId], 'timestamp').reverse()
const totalVolume = toKSM(getVolume(nfts))
const buys = nfts.length
const latestSale = nfts[0]
const medianIdx = Math.floor(buys / 2)
return {
id: idx + 1,
collectionId: colId,
name: collection.name,
totalVolume,
buys,
latestSoldSize: toKSM(latestSale.meta),
latestSoldTime: formatDistanceToNow(new Date(latestSale.timestamp)),
medianDate: formatDistanceToNow(new Date(nfts[medianIdx].timestamp)),
nfts,
}
Currently we do not store any data about transaction hash.
Thus it make sometimes hard to reference something important.
Store variable hash
in the Event
on every indexer
Extend unwrap
function :)
No response
Yes! it's blocker for kodadot/nft-gallery#3931
on explorer or on collection detail?
we can have it on both.
On explorer I need to write resolver or do other magic
Originally posted by @vikiival in kodadot/nft-gallery#5453 (comment)
Metadata
's atribute
into new entity.handleMetadata
to handle it correctlyif (total == supply || every 1000 blocks)
recalculate rarity per collectionThis is a simple resolver that should return
Array<string>
of unique addresseswe should dynamically decide which events should be visible based on schema
query availableEventsByNftId($id: String) {
list: eventList(id: $id)
}
SQL for this would look like
SELECT DISTINCT e.interaction FROM event e WHERE e.nft_id = $1
We want to know the floor
price of the TokenEntity
However the current floor on CollectionEntity is not correct
Therefore we will try different approach:
type TokenEntity @entity {
cheapest: NFTEntity
}
as I think would be easier to update :)
type CollectionEntity @entity {
sold: Int!
}
When nft is bought for the first time #35 it should increate the value
function inc(value: number | undefined) {
return (value || 0) + ONE
}
function dec(value: number) {
return value - ONE
}
type TokenEntity @entity {
id: ID!
blockNumber: BigInt
collection: CollectionEntity
nfts: [NFTEntity!] @derivedFrom(field: "token")
hash: String! @index
image: String
media: String
name: String @index
updatedAt: DateTime!
createdAt: DateTime!
count: Int!
}
The point of TokenEntity is to natively group by NFTs from same collection And same image under one entity with this we can do search, filter super easy. However this require to rapidly change schema
Id should be generated as md5(collection-cid)
?
Name would be either taken from collection of first NFT name stripped from #{number} or _{number}
Also we need to add
type NFTEntity @entity {
token: TokenEntity!
}
> You still need some way how to make refference for the picture ๐
yeah, I was thinking for some off-chain config.json for collections till we will be putting it on-chain
Originally posted by @yangwao in kodadot/nft-gallery#5286 (comment)
The hack would be to spawn system.remarkWithEvent
and extend logic for RMRK and BSX indexers
I would like to spin a first thing in the loligo package that should be isLewd as implemted in:
the Task:
occur/exist
.tstype UserTally @entity {
id: ID!
blockNumber: BigInt
updatedAt: DateTime!
createdAt: DateTime!
created: Int!
minted: Int!
burned: Int!
flipped: BigInt!
transfers: Int!
collected: Int!
}
if we want to do something historical:
type TallyEntity @entity {
id: ID!
count: Int!
createdAt: DateTime!
}
Now to get the first index we do another magic
const nextId = Math.max(lastIndexUsed + 1, collectionAlreadyMinted + 1)
it would be nice to return the index that could be used for particular collection
could be also part of #1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.