hikari-py / hikari Goto Github PK
View Code? Open in Web Editor NEWA Discord API wrapper for Python and asyncio built on good intentions.
Home Page: https://docs.hikari-py.dev/en/latest/
License: MIT License
A Discord API wrapper for Python and asyncio built on good intentions.
Home Page: https://docs.hikari-py.dev/en/latest/
License: MIT License
Stateless cache should return empty resources instead of raising.
Methods and properties should be added to guild models where appropriate.
These may take the shape of the following, but this is just a recommendation and should be at the developer's discretion.
Also, these should be tested and fully documented.
GuildWidget#fetch_channel
Member
#fetch_user
#fetch_roles
#ban
#kick
#add_roles
#remove_roles
MemberPresence
#fetch_user
#fetch_member
GuildIntegration
#fetch_user
#fetch_member
(?)#sync
(?)GuildMemberBan
GuildBan
#unban
GuildPreview
#fetch_guild
Guild
#edit
#ban
#kick
#unban
#fetch_owner
#fetch_afk_channel
#fetch_roles
async iter#fetch_emojis
#fetch_widget_channel
#fetch_system_channel
#fetch_rules_channel
#fetch_public_updates_channel
#update_voice_state
$text_channels
$voice_channels
$categories
$news_channels
$store_channels
#add_channel
#add_category
(redundant?)#remove_channel
#remove_category
(redundant?)Requires #105 to be implemented first if it is going ahead. Otherwise these will become private fields with properties.
Properties for datetimes can be converted to be stored as pure integers, since we always assume UTC. This should reduce memory usage for each guild/member/etc that exists.
The accessor can then build a value on the fly instead.
I would like to move most properties on models across to use accessor methods instead of properties. This will probably also involve making most fields private and creating getters for them.
The reason for this is that it allows me to hide whether an operation is using a cached value on the model or whether it is performing a cache lookup behind the method interface. As well as improving consistency elsewhere, it will ensure that introspecting a model instance does not perform a lot of slow operations based on the execution of properties that make cache calls internally.
This would either cover changing just cache calls, or changing all field accessions:
event.channel.id
-> event.get_channel().get_id()
This will require at least a partial rewrite of most model-bound tests.
I am still 50-50 as to whether I want to go ahead with this, since it will make stuff more verbose for people switching from other frameworks, and may be considered to be messy, but at the same time the inconsistency between "what is a field and what is a method" which may be messy currently due to Discords ™️ design should be reduced/removed.
Using accessors in Python isn't considered an antipattern per-se either. Many modules follow that approach.
Links to https://discordapp.com/channels/574921006817476608/577602779410071575/749746748615950539
We should look to change %s logging strings to format-string style ones.
Might need to make a custom Logger that forces the formatter style to prevent it conflicting with other loggers that are using %s style.
stateless=False
in the args for the bot)Traceback provided by @Chubonyo
E 2020-08-20 01:43:48,154 asyncio: Task exception was never retrieved
future: <Task finished name='gateway shard 0 dispatch GUILD_DELETE' coro=<EventManagerBase.consume_raw_event() done, defined at C:\Users\{user}\Documents\Chub bot project\.venv\lib\site-packages\hikari\impl\event_manager_base.py:81> exception=KeyError(270904126974590976)>
Traceback (most recent call last):
File "C:\Users\{user}\Documents\Chub bot project\.venv\lib\site-packages\hikari\impl\event_manager_base.py", line 89, in consume_raw_event
await callback(shard, payload)
File "C:\Users\{user}\Documents\Chub bot project\.venv\lib\site-packages\hikari\impl\stateful_event_manager.py", line 188, in on_guild_delete
self._cache.clear_members_for_guild(event.guild_id)
File "C:\Users\{user}\Documents\Chub bot project\.venv\lib\site-packages\hikari\impl\stateful_cache.py", line 1529, in clear_members_for_guild
cached_users = {user_id: self._user_entries[user_id] for user_id in cached_members}
File "C:\Users\{user}\Documents\Chub bot project\.venv\lib\site-packages\hikari\impl\stateful_cache.py", line 1529, in <dictcomp>
cached_users = {user_id: self._user_entries[user_id] for user_id in cached_members}
KeyError: 270904126974590976
hikari v2.0.0.dev69
KeyboardInterrupt
I 2020-08-29 12:05:36,624 hikari: single-sharded configuration -- you have started 0/1000 sessions prior to connecting (resets at 29/08/20 12:05:36 CEST)
I 2020-08-29 12:05:37,026 hikari.gateway.0: received HELLO, heartbeat interval is 41.25s
I 2020-08-29 12:05:37,336 hikari.gateway.0: shard is ready [session:3855c22c6cc95cf1cc397bc47a712a0b, user_id:649186216436039680, tag:Hikari Testing#6464]
^CI 2020-08-29 12:05:50,249 hikari: received signal to shut down client
I 2020-08-29 12:05:50,250 hikari: stopping 1 shard(s) # <--------- 1
I 2020-08-29 12:05:50,250 hikari.gateway.0: received request to shut down shard
W 2020-08-29 12:05:50,250 hikari.gateway.0: gateway client closed, will not attempt to restart
I 2020-08-29 12:05:50,252 hikari: stopping 1 shard(s) # <--------- 2
I 2020-08-29 12:05:50,253 hikari: The gateway client has been closed
If you were to add this to your bots code:
@bot.listen()
async def dying(event: hikari.StoppingEvent):
print("stopping")
@bot.listen()
async def dead(event: hikari.StoppedEvent):
print("died")
You would see this:
I 2020-08-29 12:11:31,251 hikari: single-sharded configuration -- you have started 1/1000 sessions prior to connecting (resets at 30/08/20 12:05:37 CEST)
I 2020-08-29 12:11:31,602 hikari.gateway.0: received HELLO, heartbeat interval is 41.25s
I 2020-08-29 12:11:31,865 hikari.gateway.0: shard is ready [session:9b3741856b9ac41c613f1644f2f58678, user_id:649186216436039680, tag:Hikari Testing#6464]
^CI 2020-08-29 12:11:32,829 hikari: received signal to shut down client
I 2020-08-29 12:11:32,829 hikari: stopping 1 shard(s) # <--------- 1
stopping # <--------- 1
I 2020-08-29 12:11:32,829 hikari.gateway.0: received request to shut down shard
W 2020-08-29 12:11:32,830 hikari.gateway.0: gateway client closed, will not attempt to restart
I 2020-08-29 12:11:32,831 hikari: stopping 1 shard(s) # <--------- 2
died # <--------- 1
stopping # <--------- 2
died # <--------- 2
I 2020-08-29 12:11:32,832 hikari: The gateway client has been closed
hikari v2.0.0 HEAD
located at /home/davfsa/coding/test-bots/hikari
CPython 3.8.2 GCC 9.3.0
Linux carbono 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64
None
logwood
appears to be a small library that piggybacks on top of Python's logging internals, but allows the use of faster and simpler logging.
We should look into whether this is a better alternative to the current logging system. If it is, we should look into migrating it.
=================================== FAILURES ===================================
_______________________ TestMRIMutableMapping.test_copy ________________________
self = <tests.hikari.utilities.test_mapping.TestMRIMutableMapping object at 0xffffa41dc850>
def test_copy(self):
raw_map = {
"floom": (0.4312, "buebue"),
"bash": (0.65234, "bunny_time"),
}
mock_map = mapping.MRIMutableMapping(raw_map, expiry=datetime.timedelta(seconds=4523412))
result = mock_map.copy()
assert result is not mock_map
assert isinstance(result, mapping.MRIMutableMapping)
> assert result == {"floom": "buebue", "bash": "bunny_time"}
E AssertionError: assert <hikari.utili...xffffa41dce50> == {'bash': 'bun...om': 'buebue'}
E Use -v to get the full diff
mock_map = <hikari.utilities.mapping.MRIMutableMapping object at 0xffffa41dc8b0>
raw_map = {}
result = <hikari.utilities.mapping.MRIMutableMapping object at 0xffffa41dce50>
self = <tests.hikari.utilities.test_mapping.TestMRIMutableMapping object at 0xffffa41dc850>
tests/hikari/utilities/test_mapping.py:133: AssertionError
----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name
EventStream.__del__ at 0xffff7d6b2040>
Traceback (most recent call last):
File "/home/travis/build/nekokatt/hikari/hikari/utilities/event_stream.py", line 208, in __del__
_LOGGER.warning("active %r streamer fell out of scope before being closed", self._event_type.__name__)
AttributeError: _event_type
Seems we can implement custom logger levels on the logging API, although not advised.
We may benefit from implementing a TRACE level for writing debugging payloads, preventing payload spam from DEBUG logs.
Allow us to do event.author
instead of event.message.author
on message create events.
We need to make ciso8601 optional, since it cannot be installed on Windows if you lack c++ redist currently...
As discussed in the discord server, typing.cast
should be removed and replaced with assert
ions where possible and, if not, use # type: the_type
Numpy docstrings support an Other Parameters
list to use along side the Parameters
one. I think it would be a nice touch to separate optional arguments away from mandatory ones using a second list.
Support no intents again because of Discord's U-turn.
Methods and properties should be added to channels.py classes.
These are just general ideas, and should be down to the discretion of the developer, specifically.
Also these should be fully documented and tested as usual.
TextChannel
fetch_history
last_message
typing
async context managertrigger_typing
send
fetch_message
GuildChannel#fetch_parent
GuildChannel#add_overwrite
GuildChannel#remove_overwrite
GuildChannel#edit_overwrite
GuildChannel#edit
GuildChannel#delete
GuildChannel#fetch_guild
GuildTextChannel#fetch_pins
GuildEmoji#fetch_roles
should be an async iterator, ideally, as it is many API calls?The following discussion from !47 should be addressed:
@nekokatt started a discussion: (+3 comments)
Any time frame on remaining time before this is ready roughly?
What else is there to do?
The cache probably still has bugs and it isn't complete for sure but there's nowhere where it'd obviously raise a ton of errors when enabled (i.e a raise NotImplementedError or missing) so it should be mergable if those other threads are fine
As far as stuff that needs to be done but I'm saving for another pr goes, we need performance improvements, intent awareness/features, test coverage (Inc a double check of how it handles hard refs), some small-consistency changes, configurability, statistics methods (e.g get count and maybe get guild ids for specific entity
pool) and consilidation around how the cache is handled by the event listeners (I.e do we nuke it on a new session) off the top of my head.
hikari.utilities.date
to hikari.utilities.time
General ideas include:
MessagePinEntryInfo#fetch_channel
MessagePinEntryInfo#fetch_message
MessageDeleteEntryInfo#fetch_channel
MemberMoveEntryInfo#fetch_channel
AuditLogEntry#fetch_user
These should be tested and documented.
CPython3.10 now has published dev docs, so it is time to ask the docker team for the Python image to make us an image for Python3.10 in their pipeline.
After this is done, we should add it to the CI.
In applications.py
, implement:
Application#fetch_guild
Application#fetch_guild_preview
These should be documented and tested.
The following discussion from !47 should be addressed:
@nekokatt started a discussion: (+2 comments)
Will the REST API return an Unavailable guild ever?
If not, we can probably ditch this model entirely, the events no longer use it as of !66, they just store
Sequence[Snowflake]
.
A new event for when guilds have been loaded and are actually ready.
Incorrect number returns from the cache being fixed. eg: what?!?!?!1 my bots in 3 guilds not 1?!?!?!
new event? idk, ask espy ;P
import hikari
bot = hikari.Bot(token=BOT_TOKEN)
@bot.listen()
async def kys(event: hikari.MessageCreateEvent):
if event.message.content == "!kys" and event.message.author.id == OWNER_ID:
await bot.close()
bot.run()
2.0.0.dev76
We could have a look at how other libs do it and make it more optimized and user friendly
Serenity (Rust): https://docs.rs/serenity/0.9.0-rc.0/serenity/model/index.html
This method would take a guild ID and return the bot's member object in the guild with the given ID. Implementation could be done using existing methods or maybe snab has a better idea. It's a pretty easy thing to do manually but this method would just make it a bit more user friendly.
Could be something like this:
def get_own_member(self, guild_id: hikari.Snowflake) -> hikari.Member:
return self.get_member(guild_id, self.get_me())
Because we no longer store member info, etc on a GatewayGuild object, but instead in the cache, we now have the issue that stateless bots cannot retrieve information within the same payload that was received such as member info.
Ideally we should provide a mechanism to allow this, probably by providing a _StatelessGatewayGuild override that stores the raw data as fields rather than in a cache.
Class/instance attributes should follow the format in the events modules for consistency and accessibility.
Stuff in hikari.events
now has the correct style of docstring - everything that doesn't just return None has a Returns
, and stuff uses an imperative mood i.e. . This includes attributes (foo: Bar = attr.ib()
) which have the """docstring"""
beneath it.
Almost everything in hikari.models is far less cohesive, often starting weird sentences like Whether your foo is enabled or not
instead of "Return builtins.True
if foo is enabled."
This issue is to sort all of that documentation out and make it consistent.
nox -spdoc
to generate the html docs in public/hikari
nox -sflake8
and nox -sreformat-code
to ensure formatting and wording is correct (this doesn't check attribute docstrings currently, so be careful).KeyboardInterrupt
Bot goes offline
Bot shows the last status before it shutdown
hikari v2.0.0 HEAD
located at /home/davfsa/coding/test-bots/hikari
CPython 3.8.2 GCC 9.3.0
Linux carbono 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64
None
Flatten the api
package into the root package.
api
to hikari
.Add typing
as solid backport dependency, like we do with mock.
This will reduce chance of bugs between Python implementations, as we will be able to make the most of bugfixes without upgrading the version of Python we target.
implement DM channel lookup by ID
This should aid places such as PrivateMessageDeleteEvent.channel where we only have the DM channel's ID without a user
TBD
I would like to start enforcing that tests pass flake8.
Seems the TypingEvent classes are missing user property overloads.
We should add that probably.
We should add some synthetic fields on models where we can provide that information based on the context we are receiving the model.
All synthetic fields should have a note:
!!! note
This is a synthetic field, meaning that discord doesn't send us
this information explicitly, we are just able to get it based on
the context.
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host discord.com:443 ssl:default [Connect call failed ('162.159.135.232', 443)]
E 2020-08-29 12:14:37,163 asyncio: Task was destroyed but it is pending!
task: <Task pending name='Task-5' coro=<RESTBucketManager.gc() running at /home/davfsa/coding/test-bots/hikari/impl/buckets.py:430> wait_for=<Future cancelled>>
E 2020-08-29 12:14:37,163 asyncio: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fc3e06aa0a0>
E 2020-08-29 12:14:37,163 asyncio: Task was destroyed but it is pending!
task: <Task pending name='Task-6' coro=<Event.wait() running at /usr/lib/python3.8/asyncio/locks.py:309> wait_for=<Future cancelled> cb=[_release_waiter(<Future cancelled>)() at /usr/lib/python3.8/asyncio/tasks.py:429]>
hikari v2.0.0 HEAD
located at /home/davfsa/coding/test-bots/hikari
CPython 3.8.2 GCC 9.3.0
Linux carbono 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64
None
hikari.api.rest.RESTClient is missing a LOT of documentation still, and should be completed.
Pdoc3 V0.9 got released about 13h ago (at the time of writing this) which includes the lunr.js search that I madr.
This should be tested that it still builds and the search bar works correctly.
Write all the missing tests
This issue is more of an epic, this will consists of a lot of small prs
This includes removing all the checks if intents is None (and typehints) since they are redundant. This can only be done after October 6, 2020
EnumMeta in the standard lib is slow, and during deserialization we invoke this a lot.
We should have a custom impl such that;
if TYPE_CHECKING:
EnumMeta = enum.EnumMeta
Enum = enum.Enum
...
else:
class EnumMeta(...):
...
class Enum(...):
...
These should have a similar interface but be optimised where possible for speed.
Benchmarking is a must.
add_reaction
- adds a reaction as the bot.remove_reaction
- removes a reaction as the bot.fetch_user
fetch_message
fetch_channel
fetch_guild
^-- same applies for removal events
KeyboardInterrupt
Bot shuts down
Bot stays online
oooo o8o oooo o8o 光 v2.0.0 [HEAD]
`888 `"' `888 `"' © 2020 Nekokatt, licensed under MIT
888 .oo. oooo 888 oooo .oooo. oooo d8b oooo CPython 3.8.2
888P"Y88b `888 888 .8P' `P )88b `888""8P `888 running on: 5.4.0-45-generic Linux x86_64
888 888 888 888888. .oP"888 888 888 Installed at: /home/davfsa/coding/test-bots/hikari
888 888 888 888 `88b. d8( 888 888 888 Documentation: https://nekokatt.github.io/hikari
o888o o888o o888o o888o o888o `Y888""8o d888b o888o Support: https://discord.gg/Jx4cNGG
I 2020-09-02 08:59:28,130 hikari: this application is stateless, cache-based operations will not be available
I 2020-09-02 08:59:28,693 hikari: single-sharded configuration -- you have started 0/1000 sessions prior to connecting (resets at 02/09/20 08:59:28 CEST)
I 2020-09-02 08:59:28,997 hikari.gateway.0: received HELLO, heartbeat interval is 41.25s
I 2020-09-02 08:59:29,201 hikari.gateway.0: shard is ready [session:53d3a6ac33cbf7fb2095bb1c43546588, user_id:649186216436039680, tag:Hikari Testing#6464]
^CI 2020-09-02 09:07:42,225 hikari: received signal to shut down client
I 2020-09-02 09:07:42,226 hikari.gateway.0: received request to shut down shard
I 2020-09-02 09:07:42,228 hikari.gateway.0: gateway client closed, will not attempt to restart
I 2020-09-02 09:07:42,441 hikari.gateway.0: received HELLO, heartbeat interval is 41.25s
W 2020-09-02 09:07:42,551 hikari.gateway.0: invalid session, so will attempt to reconnect with new session in a few seconds
I 2020-09-02 09:07:46,388 hikari.gateway.0: received HELLO, heartbeat interval is 41.25s
I 2020-09-02 09:07:46,677 hikari.gateway.0: shard is ready [session:1b4578f4850c678c90c88dbec76e6817, user_id:649186216436039680, tag:Hikari Testing#6464]
# Me trying to shut it down again
^CI 2020-09-02 09:07:47,754 hikari: received signal to shut down client
^CI 2020-09-02 09:07:49,105 hikari: received signal to shut down client
^CI 2020-09-02 09:07:49,552 hikari: received signal to shut down client
^CI 2020-09-02 09:07:49,754 hikari: received signal to shut down client
^CI 2020-09-02 09:07:58,969 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,137 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,312 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,475 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,625 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,799 hikari: received signal to shut down client
^CI 2020-09-02 09:07:59,985 hikari: received signal to shut down client
^CI 2020-09-02 09:08:00,151 hikari: received signal to shut down client
hikari v2.0.0 HEAD
located at /home/davfsa/coding/hikari/hikari
CPython 3.8.2 GCC 9.3.0
Linux carbono 5.4.0-45-generic #49-Ubuntu SMP Wed Aug 26 13:38:52 UTC 2020 x86_64 x86_64
I have not been able to reproduce this again. Maybe we should suppress all other closures when we receive the signal to shutdown the bot?
Hopefully with the release of Cython 3 at some point, and cython/cython#3691 hopefully being merged soon (thus giving support for the := operator), we should be able to start producing Cythonized artifacts for x86.
I believe the main issues before when we tried this were around:
from __future__ import annotations
was not supported. I know this is now supported.Generic[T]
did not work properly. I am pretty sure this works now.:=
was not supported (see the above PR).This should give a nice performance boost where users choose to opt into the experimental accelerated builds :) (Cython usually gives at least 10 times speed increase over pure python from experience).
Trying to handle the chunk events for a large guild too quickly using the event dispatcher's wait_for method leads to the event manager trying to handle the wait_for listener's generated future at an invalid state leading to a traceback being raised to logging and some events being missed.
import hikari.utilities.date
guild_id = LARGE_GUILD_ID
nonce = hikari.utilities.date.uuid()
await bot.shards[0].request_guild_members(guild_id, nonce=nonce)
results = []
def predicate(event):
return event.nonce == nonce
while True:
try:
chunk = await bot.dispatcher.wait_for(
hikari.events.shard_events.MemberChunkEvent, 5, predicate=predicate
)
results.append(chunk.chunk_index)
except asyncio.TimeoutError:
break
print(results)
the following to be printed
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]
[0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 13, 15, 17, 20, 22, 24, 26, 29, 31, 33, 36, 38, 40, 42, 45, 47, 49, 51, 54, 56, 58, 60, 62]
is printed along with the following traceback making it's way to logging for each missed chunk event.
E 2020-08-23 03:34:04,967 asyncio: Task exception was never retrieved
future: <Task finished name='gateway shard 0 dispatch GUILD_MEMBERS_CHUNK' coro=<EventManagerBase.consume_raw_event() done, defined at C:\Users\Snab\PycharmProjects\hikari\hikari\impl\event_manager_base.py:81> exception=InvalidStateError('invalid state')>
Traceback (most recent call last):
File "C:\Users\Snab\PycharmProjects\hikari\hikari\impl\event_manager_base.py", line 89, in consume_raw_event
await callback(shard, payload)
File "C:\Users\Snab\PycharmProjects\hikari\hikari\impl\stateful_event_manager.py", line 252, in on_guild_members_chunk
await self.dispatch(event)
File "C:\Users\Snab\PycharmProjects\hikari\hikari\impl\event_manager_base.py", line 247, in _test_waiter
future.set_result(event)
asyncio.exceptions.InvalidStateError: invalid state
hikari v2.0.0 HEAD
located at C:\Users\Snab\PycharmProjects\hikari\hikari
CPython 3.8.2 MSC v.1916 64 bit (AMD64)
I would like to spend some more time testing REST only support to verify that it is in fact usable and not broken internally, as I don't believe we have any consumers of that bit yet (that I know of)
discord/discord-api-docs@960a8ea
Missing documented REST api coverage
TBD
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.