Comments (2)
This problem is not specific to the HTTP server implementation. We had the same problem in ScyllaDB in other non-http servers, which like the HTTP server listen()ed
and accept()ed
connections in all cores in parallel. The bad port is detected as expected on shard 0, but other shards result in the assertion failure shown above, and cause the whole application to crash. The problem should probably be fixed in the context of listen()
and/or accept()
, not the HTTP server. I'm removing the HTTP server label from the issue.
from seastar.
The code in
src/net/posix-stack.cc
has a strangeassert(i.second)
, with no explanation what it means, and this is the assert that fails.
The assertion means, that the code tried to emplace some new object to wait for connection, but emplace returned that that object was already there. See below for why the object was expected to be new and why it turned out that it wasn't.
This assert was introduced by @gleb-cloudius almost a decade ago, in commit 19addf6. The commit message explains that "only one thread opens listen socket and runs accept(). Other threads emulate listen()/accept() by waiting for connected socket from the main thread.". I'm not 100% sure that this commit message is actually up to date, because I was under the impression that today all cores accept() connections - and send them to other cores. But I'm not sure.
Currently only one core accepts sockets for real, other cores wait on sockets from that core via "ap-server" thing:
network_stack_entry register_posix_stack() {
return network_stack_entry{
"posix", std::make_unique<program_options::option_group>(nullptr, "Posix"),
[](const program_options::option_group& ops) {
return smp::main_thread() ? posix_network_stack::create(ops)
: posix_ap_network_stack::create(ops);
},
true};
}
the main_thread()
returns true to only one shard's thread
This also raises another question: Is it important that http_server::listen() be called first on core 0, and only then on other cores? Right now I call all of them in parallel, and luckily it does run on core 0 first, but what if it runs first on core 1, before core 0?
This is how it works.
First, the httpd::listen()
calls seastar::listen()
and .accept()
on the returned server_socket
. On shard 0 this maps to posix_network_stack::listen()
+ posix_server_socket_impl::accept()
. On non-0 shards this maps to posix_ap_network_stack::listen()
+ posix_ap_server_socket_impl::accept()
. The ap-socket's accept code emplaces a "new object" mentioned above into sockets
map to listen for connections coming from main thread. The sockets
map is std::unordered_map<std::tuple<int, socket_address>, promise<accept_result>>
, where int
is the protocol (PF_...
value). So it maps socket address to a promise which will be resolved into connected socket later.
Second, yet another httpd::listen()
is called and it does the very same thing -- calls seastar::listen()
and .accept()
on the returned socket and both map to the same pair of calls as in first step. On non-main thread the .accept()
call, which maps to posix_ap_server_socket_impl::accept()
, tries to emplace one more "new object" mentioned above into sockets
map to listen for connections coming from main thread, but the key it emplaces, is already there after the first https::listen()
! It's already there, because socket_address
part is the same for both -- first and second invocations. Thus the assertion.
from seastar.
Related Issues (20)
- cgroups v1 isn't supported HOT 8
- use undeprecated functions in c-ares HOT 2
- Cross-shard realloc shrink crashes
- Additional data when segfaulting
- Dynamic metric aggregation
- create_scheduling_group / scheduling_group_key_create not exception safe when SG key data constructor throws
- add a unit test for json2code
- create_scheduling_group / scheduling_group_key_create are not safe to run in parallel HOT 1
- add test for prometheus (aggregation)
- [project unrelated] Question about solving __common_cmp_cat duplicate definition problem HOT 1
- High lowres clock drift > 1s in debug mode HOT 4
- add an integration test with scylla
- the workflow of "Tests (clang++-18, C++23, release, --enable-dpdk) fails due to dpdk build failure HOT 3
- seastar allocator causes infinite recursive call if seastar is compiled as a shared library
- io_uring reactor backend not available when importing and using seastar as a module HOT 2
- circleci CI was removed but it's still being reported HOT 9
- should revert ba395a22e1951a1572be2a570092524d91996c67 once clang 18.1.6 is in f40
- Update the benchmark HOT 1
- Integrate with industry-standard benchmarks
- Add `sharded<>::invoke_on_all()` const overload HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from seastar.