dlang / project-ideas Goto Github PK

View Code? Open in Web Editor NEW

36.0 36.0 12.0 50 KB

Collection of impactful projects in the D ecosystem

project-ideas's People

Contributors

Stargazers

Watchers

Forkers

gallafrancesco thewilsonator wilzbach rajat-np laeeth rodrigoieh razvann7 mdparker-dlang-forks seanpm2001 menjaraz dchidindu5

project-ideas's Issues

ref counting

build a reference counting solution for D

Description

It would be great and helpful if a crypto module can be included in std.

What are rough milestones of this project?

To include some basic crypto methods in the standard library of D language

Recommended skills

moderate D language experience
cryptography knowledge

Point of Contact

@vikasgola

References

https://forum.dlang.org/thread/[email protected]

Create a CI or other infrastructure for measuring D's progress and performance

Description

We need to be able to measure how the compiler and the standard library perform over time. This project would create a CI, or other infrastructure, that would monitor metrics like compile-time, compiler's memory consumption, artifact's runtime performance, artifact's memory consumption, artifact's binary size, etc., so that so we can measure, quantitatively, D's progress over time.

It could even be expanded to include other non-performance metrics such as numbers of contributors, top contributors, lines of code changed per unit time, etc.

What are rough milestones of this project?

The project implementer is encouraged to propose their own ideas and vision for this project.

How does this project help the D community?

This project would help prioritize work and measure, quantitatively, D's progress as a project.

Recommended skills

The project implementer should have some skills benchmarking software
The project implementer should have some skills creating and using charts and other forms of data visualization to convey information at a glance

Point of Contact

@JinShil can be contacted to further elaborate on the project's goals. However, there is significant artistic license granted to those willing to do the work.

References

Mozilla's Are We Sim Yet
D's AutoTester Metrics - These are not performance metrics but they are an example of kinds of visuals that would help D, as a project, measure its progress.

Persistent Data Structures

There is an ongoing effort to reduce the reliance of Phobos on the GC
but the GC is not going anywhere given that it has many advantages over
manual memory management. The community would greatly benefit from a
wide range of functional programming techniques that have their roots in
the usage of immutable and persistent data structures that heavily rely
on the GC. Persistent Data Structures are heavily used in
Scala
and Clojure and they
allow the programmer to reason about interactivity, sharing and
parallelism.

The focus of this project would be to create a varying range of
persistent data structures (sets, vectors, maps, strings, trees, lists,
etc...) that respects the range interface and the semantics of Phobos.

Related Work:

DUB: D's package manager

[https://github.com/dlang/dub DUB] - D's package manager - is one of the central infrastructure tools in the D world as a convenient build and package management tool for the D community.
With D gaining more and more popularity, it's also a key tool to guarantee D's further adoption.
Thus there are many great ideas to improve DUB:

https://github.com/dlang/dub/wiki/DEP6 git support
https://github.com/dlang/dub/wiki/DEP2 separate compilation and caching
https://github.com/dlang/dub/wiki/DEP3 extensibility through DUB
https://github.com/dlang/dub/wiki/DEP4 control flow directives
https://github.com/dlang/dub/wiki/DEP5 build system support for C/C++
support dub install for distributions
support building a {snap, Debian, Arch, ...} package out of DUB package

Any of these listed DUB Enhancement Proposals (DEPs) could form the basis of a GSoC Project.

'''Update:''' A very detailed list of ideas for improving Dub can be found in this Forum thread

iOS support in LDC

Mobile is becoming, or rather is already, the biggest software platform.
While LDC already supports the Android platform there's no official
support for iOS. The project would consist of adding support for the iOS
platform to the LDC compiler, the D runtime library and the D standard
library. The goal would be to get support for this platform upstreamed
to the LDC main project.

A lot of existing work has been done but has never been completed or
upstreamed: (https://github.com/smolt/ldc-iphone-dev)

Proposed Project Mentors: TBA (LDC dev team)

Point of Contact: @jacob-carlborg

Are we @nogc yet?

Description

It's common to see potential D users comment that large parts of the standard library aren't usable with the GC. It's difficult to counter that assertion without any data - I don't think anyone knows how much of Phobos can be used from @nogc code, nor could we point people to a resource for more information.

Similarly to Python's former wall of shame that got renamed to the wall of superpowers, it would be great if D had a webpage titled "Are we @nogc yet", preferably with a code-coverage-style visualisation of Phobos. At the very least, a list of functions/structs/classes that are @nogc compatible.

What are rough milestones of this project?

Annotation of Phobos unittests with @nogc if possible
A tool to extract the information of what Phobos code is called/covered by such unittests
A web page to visualise the data

How does this project help the D community?

It helps the community by helping to dispel the myth that the GC is central to D programming.

Recommended skills

Webpage design and implementation.

Rating

Easy

Project type

Low hanging fruit, Fun

Point of Contact/Possible mentors

@atilaneves @maxhaton

MLIR support for LDC

Description

Multi-Level Intermediate Representation (MLIR) was recently announced as higher level IR for semantic aware transformation for optimisations.
It is expected to be integrated into the LLVM project in the near future.
MLIR represents an opportunity to leverage advanced optimisations at a higher level than that available in LLVM and target other systems such as .

What are rough milestones of this project?

Become familiar with the LDC & MLIR codebases.

Compile D through the LLVM dialect of MLIR.

Port some basic LDC specific optimisations (e.g. GC2Stack) as a D Dialect of MLIR to verify the functioning of optimisation with MLIR.

Expose the ability to utilise other MLIR dialects such as Affine (for polyhedral optimisations) and Vector

Investigate the ability to generate XLA with MLIR for use with Tensorflow.

How does this project help the D community?

Allow the utilisation of advanced optimisations for D code and potentially generate XLA for use with Tensorflow.

Recommended skills

C++ (insofar as the style that is used by LLVM, nothing too arcane needed)
Some Experience with compilers (ASTs, SSA-IRs, IR transformations)
Experience with LLVM is an added bonus.
Experience with D nice, but not necessary.

What can students expect to get out of doing this project?

Get experience with cutting edge high level compiler optimisation techniques and apply them to a production compiler.

An opportunity to present the work done at EuroLLVM (May 2020 Europe) (or LLVM Developers Meeting October 2020 San Jose) and possibly other conferences (e.g. Tensorflow Dev Summit).

Point of Contact

@thewilsonator (Nicholas Wilson)

Other references

f18/flang design docs considering using MLIR

Improve DUB user experience

Description

The status quo of Dub is far from ideal. There are many low-hanging fruits that could be picked to improve the user experience and infrastructure for the entire D community.

What are rough milestones of this project?

High-value:

parallel fetch
dub install (dlang/dub#839)
parallel build of independent modules

Nice to have:

colored dub error messages (dlang/dub#1490)
support for git dependencies (dlang/dub#1403)
dub watch (dlang/dub#446)

How does this project help the D community?

DUB is a critical core piece of the D community. Improving Dub will automatically improve the experience for all D users and its currently 1500 packages.

Recommended skills

Medium experience with D
Experience with Linux (for the "dub install" and "dub watch" sub-projects)

Point of Contact

Martin Nowak (@MartinNowak)
Sönke Ludwig (@sludwig)
Sebastian Wilzbach (@wilzbach)

References

Prior discussions:

Integrate multiple D commandline utilities into one

Description

A single command that will bridge multiple D related applications like :

dmd / rdmd / ldmd2 / gdmd2
dfmt
dfix
dub
dscanner
dustmite
ddemangle
drepl
etc.

Similar to busybox or the go command (yes i know i said the g-word :-)). It's a bit of a shame we would not be able to use do i guess. Until a better name comes along, assume we use the name do for now, then sub commands could be:

do compile
do build
do test
do repl | do cli
do profile
do initproject
do umbrella
do fix
do fmt
do scan
do mangle
do generate [dub.sdl | dub.json | cmake-d]
do convert [dub.sdl | dub.json | cmake-d]
etc

In the first interation it could just be a simple frontend to these other commands. As time progresses deeper integration would be possible, for example:

parallelizing builds with smart compiler integration. Whereby dub could provide multiple paths/modules to be compiled and the compiler could generate backpressure when needed.
have the compiler spit out suggestions to counteract deprecation's, which would in turn could generate a diff/patch generated by dfix and dfmt.

How does this project help the D community?

In the beginning, it would simplify user interaction and allow new users to get exposed to more of the available d-tools without having to search for them. When the integration progresses, it could smooth the way for to make changes to the language without large negative impact (as in the latter example).

References

Solve dependency hell: link with more than one versions of the same project

Description

Let's see that in project X you use an XML parser of v1. This XML parser depends on a collections library with v1. Now, however, the team decides to a add a JSON parser too, but this one depends on a newer version of the collections library with v2. The result is that the team can't use the JSON parser in their project and currently the only viable strategies are:

copy/paste the collections library to a new namespace and let the JSON parser depend on this new collections namespace
try to upgrade the XML parser to v2

(1) is very cumbersome, requires a lot of manual work and makes it hard to sync up with upstream. (2) is often not possible.

Hence, the aim of this project is to allow people to use v1 and v2 of collections in parallel in their dub projects. Rust allows to fix this dependency hell problem by appending a unique suffix to each symbol of the conflicting versions. This works as long as each dependency only uses one direct version of a dependency. This is a very reasonable assumption and this approach should work for D too.

What are rough milestones of this project?

investigate solutions used in other languages or active research
create two DMD PR
- to allow it to append unique suffixes to all generated symbols (e.g. --mangle-suffix=v1.1)
- to map unique suffixes to used symbols (e.g. --mangle-suffix-map=json:v1.1)
add the respective CLI flags to all dub invocations (be sure only to do so if frontend >= X)

solve a small minimal example

only add CLI flags to dub dmd invocations when they are required (e.g. SemVer conflict)

create more test cases

open DUB PR
get DUB PR merged

make sure it passes on all CIs
address review

Optional improvements:

allow the user to completely disable this behavior and error when an unsolvable conflict has been found (= revert to current behavior)
allow to user to declare the exact semver multi-mapping in his/her's project config

How does this project help the D community?

It is a first milestone and a requirement to moving the standard library Phobos to Dub. Additionally, it will have a massive impact on the D community as any bigger D project that has a non-trivial amount of dependencies runs into a dependency hell.

Recommended skills

medium experience with D
experience with a package manager
understanding of the linking process
no fear of the DMD frontend

What can students expect to get out of doing this project?

learn about D's name mangling system
learn how D packages are built
create and integrate a high-impact project for the D community

Rating

Medium

Project Type

Infrastructure/Automation

Point of Contact/Possible Mentors

@wilzbach @maxhaton

References

https://stephencoakley.com/2019/04/24/how-rust-solved-dependency-hell

Migrate makefiles for DMD, Phobos, and druntime to D

Description

Currently most D repositories require build tools that are specific to flavors of Unix, and those tools are not portable to all platforms that D supports. For Windows specifically, users have quite a bit of difficulty setting up their environment so they can build DMD, Phobos, and druntime. But there is a solution. We have a tool that allows us to automate just about anything on all platforms that D supports: The D Programming Language. Therefore, this project proposes converting all makefiles and other build tooling for DMD, Phobos, and druntime to D.

This project has already been started. A working (though incomplete) build.d can be found in the DMD repository. It just needs someone to take it to completion.

What are rough milestones of this project?

Finish porting the remaining makefile targets in DMD to build.d
Write a similar build.d for druntime
Write a similar build.d for Phobos

How does this project help the D community?

The D programming language is an extremely powerful and expressive language to program in. The DMD, Phobos, and druntime repositories already require a D host compiler, so utilizing D to implement the build system will require no additional dependencies and will ultimately remove several unnecessary dependencies. Contributors working on Linux, Windows, or any other platform D supports will have the same experience when building DMD, Phobos, and druntime. There will no longer be any need for separate "How to build on Linux" instructions, "How to build on Windows" instructions, etc; there will only be "How to build" instructions.
Furthermore, D is a tool all D contributors have in common despite their background or platform of choice. Contributors will already know how to modify, compile, debug and test D programs, so it will lower the barrier to entry and learning curve for anyone modifying, enhancing, or otherwise improving D's build infrastructure and will make the D programming language development effort less dependent on specialists.

Recommended skills

Familiarity with the D programming language and the general procedures for how to build and test DMD, Phobos, and druntime
Familiarity with makefiles

Point of Contact

This project was proposed by @JinShil and can be contacted for further information about the project.

References

DMD's build.d implementation

Typelib support inside dmd-fe

Description

Typelib is a fileformat used for COM interfaces. It is the compiled form that Microsoft ships IDL files in.

What are rough milestones of this project?

Able to parse Typelib files via WinAPI
Able to generate from the parsed files AST
Ship our own TypeLib files on Windows (compile the IDL files provided by Microsoft).
While this should be identical to the ones Microsoft provided, it may be best to be able to say when shipping that we didn't just copy them.

How does this project help the D community?

It gives the D community first class support for using COM classes inside of D and doesn't require massive binding files (and growing) to support it.

Recommended skills

COM
Basic understanding of the compiler, parse tree vs AST ext.

GSoC and SAoC are good targets for this project. If there is time over the project can be extended to generate IDL files so other languages can call into D code equally as well.

What can students expect to get out of doing this project?

Familiarization of "modern" Windows API's structure and a basic level of knowledge of how to generate AST inside the compiler frontend.

Point of Contact

References

Jupyter notebook D kernel

The Jupyter Notebook is an open-source web application that allows you
to create and share documents that contain live code, equations,
visualizations and narrative text. Uses include: data cleaning and
transformation, numerical simulation, statistical modeling, data
visualization, machine learning, and much more. Jupyter notebooks are
extremely popular among data scientist as they allow interactive
progress and exploring a solution step by step. With a bit of work, a
statically compiled language can be used in an interactive notebook e.g.
Cling
for C++ and gophernotes for
Go. Apart from being useful to data scientist, a Jupyter D kernel would
allow an intuitive exploration of the language for newcomers and people
starting with new libraries.

Existing works:

drepl - simple REPL for
D
PydMagic - allows to
write PyD extension in a notebook.
jupiter-wire
and jupyterd bare
kernels for interfacing interpreters to

DMD front end should be instrumented using SystemTap / Bpftrace on Linux

Description

What are rough milestones of this project?

How does this project help the D community?

Recommended skills

Point of Contact

References

std.interprocess

The goal of this project is to implement a flexible interprocess data
structure, synchronization and communication library. An equivalent
library in C++ is Boost
Interprocess.
The important goal would be to port all the features from
boost.interprocess to D allowing the creation of structs/classes on
shared memory, etc.

boost::interprocess offers the following features and it will be
challenging & achieving to have these implemented in D in a stable, easy
to use library that achieves all the aspects.

default and custom allocators to operate on shm
named IPC primitives: shm, semaphores, spinlock, mutex, condvar,
barriers
standard containers like vector, deque, list, string, set that
operates on shm
message queue on shm without relying on the kernel for data
transfer. This is a huge win for low latency apps in the order of
micro seconds
managed shared memory
IPC smart pointers

Get started by looking into the following:

Mir Project

The Mir project is developing numerical libraries for the upcoming
numeric packages of the Dlang standard library. The are numerous
projects an interested student could pursue:

University project. Your GSoC project can be combined with your
scientific research.
ndslice
Julia integration
NumPy integration
General purpose BetterC libraries
- I/O betterC Dlang library
- Async betterC Dlang library
- String and format betterC library
mir-cpuid
- ARM support
- Advanced OS specific support
mir-glas
- BLAS Level 2 subprograms
- One BLAS Level 3 subprogram. It is OK to do only one! But it
  must be faster then OpenBLAS, and not slower than Intel MKL.
- Multithreading support for BLAS Level 2
- Multithreading support for BLAS Level 3

You can get more details on projects related to Mir
here.

It's Good To Know:

To work on the Mir you should be proficient with one of:

C
C++
LLVM
Fortran
Experience with D is not essential.

To work on the Mir project requires a responsible and self-motivated
student.

Proposed Project Mentor: @9il

DPP with Linux kernel headers

Description

Razvan (@RazvanN7 ) and I worked last year with Alex Militaru on writing linux drivers in D.
At the time we tried to use DPP to #include kernel headers in D modules, but we didn't manage to get it to work on the Linux code base so we had to do everything by hand.

As you probably know from personal experience, manually translating headers to .di interfaces is a tedious, repetitive, boring and error prone (because of it repetitive and boring) task for a programmer and a waste of actual development time. This is a perfect task for automation.

We feel that adding DPP support for the Linux code base would be a great addition and a natural way of going forward for writing drivers in D.

What are rough milestones of this project?

Get familiar with DPP's code and work flow
Understand why DPP doesn't work on the Linux code base (my guess is because of GCC specific flags)
Make DPP work on failing cases (eg. make it learn about GCC specifics and how to ignore or translate them without breaking code)

How does this project help the D community?

This would be a great enabler for any developer that desires to write a Linux driver in D, and I feel that it will also be received well by the community.

Recommended skills

Willingness to learn
Determination
C, D
bonus: experience with llvm/libclang
bonus: experience with Linux build system

(If applicable, e.g. GSoC/SAoC)

What can students expect to get out of doing this project?

how to debug
how DPP works
how language interop works at the lowest level

Point of Contact

@edi33416 @RazvanN7 @atilaneves

References

Tabular data container (data frames)

Pandas, R and Julia have made data
frames
very popular. As D is getting more interest from data scientist (e.g.
eBay or
AdRoll)
it would be very beneficial to use one language for the entire data
analysis pipeline - especially considering that D (in contrast to
popular languages like Python, R or Julia) - is compiled to native
machine code and gets optimized by the sophisticated LLVM backend.

Minimum requirements:

conversion to and from CSV
multi-indexing
column binary operations, e.g. `column1 * column2`
group-by on an arbitrary number of columns
column/group aggregations

Announce dlang/projects

Description

Write an announcement of dlang/projects on the forums

What are rough milestones of this project?

How does this project help the D community?

Recommended skills

Point of Contact

@wilzbach @thewilsonator @burner @atilaneves

References

Java Native Interface (JNI) library

D's features allow to write cut down the boilerplate of wrapper
drastically. See the recently published
excel-d
for a comparison between the C++ implementation for registering a
function in Excel and the D way. Similarly a solid JNI library would
open up new many new use cases and allow the Java community to profit
even easier from D's features.

Previous work: DJni,
djvm

std.parallelism

The `std.parallelism` package has been part of Phobos for a while now,
and is in production use with many people/applications. Phobos generally
has evolved since `std.parallelism` was created and there has been
other work on fibers and co-routines in other packages within the D
milieu. It would therefore seem a good time to review the tasks system
at the heart of `std.parallelism` to see if there are improvements
that can be made or general refactorings that are possible. Or it may be
that the tasks system just needs some tweaks to improve performance. The
issue here is that n active review of `std.parallelism` is due.

Graphics library for resource constrained embedded systems

Create a 2D rasterizer, rich drawing primitives, and 2D graphics library
suitable for resource constrained embedded systems (e.g. ARM Cortex-M)
to be used in industrial controls, home appliances, medical devices,
consumer electronics, and IoT just to name a few. The end goal would be
something similar to Segger's
emWin. The
library would be used to drive LDCs similar to
https://www.adafruit.com/product/3396 Requirements:

Hardware agnostic; should simply render to a frame buffer
No dependencies (No Phobos, no C standard library, and no official D
runtime).
Consider using -betterC, but a custom minimal D runtime is also a
viable option

Related work:

LDC (LLVM-based D compiler)

The LDC project provides a portable D programming language compiler with
modern optimization and code generation capabilities. The ever-improving
feature set of LLVM opens up many opportunities with D. A few ideas for
a potential GSoC project are:

exploiting immutability of variables (notably of structs such as
string slices)
devirtualization
improve debuginfo
ASan with GC and multithreading
function multiversioning

You might also want to browse the LDC's issue
tracker for more ideas.

Proposed Project Mentors: TBA (LDC dev team, @thewilsonator )

Instrument DMD frontend

Description

It is difficult to find out why the use of many templates slows down DMD.
Profilers are not totally helpful, as they normally don't report the values past to functions.

Think prometheus + grafana for dmd.
Maybe something like

void* someFunction(int v, string templateName) {
    version(Stats) auto s = Stats!(&someFunction)(v, templateName)();
}

such that Stats logs the name of the functions a reasonable about of the arguments,
and takes the time how long the function took to execute.

What are rough milestones of this project?

Write the Stats functionality
Created PR for dmd
Find out what is slow in DMD and why

How does this project help the D community?

Should be a good first step to figure out why dmd is slow sometimes.
dmd being faster, will mean that we have to do less waiting

Recommended skills

Willingness to learn.
Frustration tolerance.

(If applicable, e.g. GSoC/SAoC)

What can students expect to get out of doing this project?

How to profile
What do profile
Some inside into how dmd works

Point of Contact

@WalterBright @burner

References

ORM mapping

SQLAlchemy is an object-relational mapper which provides "a full suite
of well known enterprise-level persistence patterns, designed for
efficient and high-performing database access, adapted into a simple and
Pythonic domain language". While there has been some attempts in
implementing an ORM in D - none of these got close to the simplicity and
usability of SQLAlchemy (see Database
Libraries). However, with D's CTFE
queries could - like std.format's strings - already be checked at
compile-time and optimizations could be be applied to the parser and
serializer.

This project would be based on the proposed std.database
abstraction and focus on
creating a general-purpose ORM on top of it.

See also: Martin Nowak's DConf16 talk about Object-Relational
Mapper

Existing work:

hunt-entity

Independency of D from the C Standard Library

An effort to decouple D from the C standard library.

GDC (GCC-based D compiler)

The GDC compiles uses GCC as a backend to utilize this powerful backend.
Recently, GDC has been accepted as GCC
frontend
and will be part of the GCC9 release. However, GDC still has lots of
tasks that the potential students could work on.

One idea would be to improve GDC's support for betterC by adding a
compiler interface to allow using a modular runtime. Right now when GDC
uses some feature (TypeInfo, asserts, unittests), it just emits a call
to an `extern(C)` function. A much better approach would be to have a
defined D API (e.g. `rt.assert.onAssertFailure(...)`). The compiler
can then check if it finds `rt.assert.onFailure`. If not, the compiler
can the simply disable that feature and provide nice error messages.
This could be done for TypeInfo, Exceptions, Asserts, Switch errors,
unit test support, etc. In the end, the vision would be to be able to
ship a compiler completely without runtime. This compiler should be able
to produce bare bones code without any further configuration. Then add a
dependency to `drt:core` in the dub configuration to get class
support, `dr:exception` for exceptions, etc. The compiler could then
automatically detect availability of these features. (Of course manually
adding -I paths would work in exactly the same way).

For more ideas, interested students are encouraged to browse GDC's
issue list or
getting in touch with the GDC development
team.

Da

Description

What are rough milestones of this project?

How does this project help the D community?

Recommended skills

Point of Contact

References

Header generation for C/C++

With the ever-improving integration between DMD and C/C++, quite a bit of tooling has been developed to generate D header files from C or C++ projects (e.g. (https://github.com/jacob-carlborg/dstep DStep) or (https://github.com/atilaneves/dpp DPP)).
However, for many projects the majority of the source code is now written in D and the reverse step is needed: an automatic way to generate C and C++ headers from a D project.
This would be very valuable for D as it could motivate inter-language projects that typically go C++-first-with-a-D-binding to work the other way.

Starting points:

std.benchmark

Ideally every function in Phobos should have its own benchmark suite
which is run for every PR to check for performance improvements or
regressions.

Additionally support for the new big O annotation
here would be nice.
Existing work: here, and
here, and here
(QuickText-like
testing).

Moving Bugzilla to Github

Description

Moving bugzilla issues to github

What are rough milestones of this project?

build automation
run in parallel
validate everything is good
shut bugzilla down

How does this project help the D community?

bugzilla is not really state of the art anymore
read more about it here
https://www.python.org/dev/peps/pep-0581
http://pyfound.blogspot.com/2019/05/mariatta-wijaya-lets-use-github-issues.html

Recommended skills

Point of Contact

References

<NG discussions, GitHub PRs, Bugzilla issues, ...>
https://forum.dlang.org/post/[email protected]

Dlang plugin for SonarQube

Description

SonarQube is a tool to manage check (DScanner) and test results (Unittests & Code Coverage). Found issues can be assigned to users and can be tracked. Additional metrics like code duplication and complexity are also be evaluated.
SonarQube nicely integrates with build pipelines and code repositories like Github. Found issue will be directly shown in Github pull requests as comments from SonarQube. Also SonarQube provides metrics as badges which can be directly integrated in markdown files.

While there is a SonarQube plugin for D (https://github.com/economicmodeling/sonar-d-plugin) it is outdated, has limited functionality and is not added to SonarQube marketplace.
As it is not available on marketplace, companies refuses to install it.

What are rough milestones of this project?

Update existing plugin to recent SonarQube version (7.x)
Add the plugin to the SonarQube marketplace
Add additional sensors (code coverage / unittests / complexity / ...)
Adapt plugin to recent changes in D (e.g. body keyword)

How does this project help the D community?

For companies, SonarQube is a great tool to keep track of their projects regarding quality and issues. All check and test results are available at a central place and can be evaluated and further be processed by other tools.

Recommended skills

Java, SonarQube

Point of Contact

References

https://www.sonarqube.org/
https://github.com/economicmodeling/sonar-d-plugin

Start tagging with 'saoc'

In the light of the recent annoucement of SAoC 2019, we should start tagging appropriate projects with the 'saoc2019' tag. Possible targets:

Finish experimental.xml

this project was part of a previous summer of code.
It was mostly finished, but lacked some polishing.

I @burner believe that not much work is needed to make it nice and polished.

Use RC-counted exceptions by default

Description

DIP1008 allows allocating and catching exceptions in @nogc code.

What are rough milestones of this project?

Make Druntime and Phobos compile with -preview=dip1008
Make all projects on the project tester compile with -preview=dip1008
Switch the flip

How does this project help the D community?

This will finally allow exceptions to be used in @nogc code and thus greatly increase the usage area of @nogc.

Recommended skills

Intermediate D knowledge
Understanding of DIP1008
Being able to build DRuntime of Phobos

Point of Contact

@wilzbach

References

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
dlang/dmd#8508 (shows what breaks when flipping the switch)

BetterC runtime

BetterC is a subset
of D which limits the feature set of D, s.t. it's even easier to
migrate from C as no Garbage Collection (GC) is used anymore. While,
this feature is popular for some projects, `-betterC` today means
still means a severe restriction in the amount of D features available
(see the BetterC specification).
This project is about making more parts of the D runtime
betterC-compatible. For example, classes could be allocated with
reference-counting instead of using the GC.

Auto-build release binaries via CIs on dlang/dmd

Description

Building DMD releases is a complicated process that at the moment only a single member of the D community understands fully. This has led to the problem that currently no 64-bit DMD executable is available as official release (despite it being built and tested on dmd's CIs) or that the sources in the release archives don't allow building DMD. DMD nightly builds currently run hidden once a day and their log is private which lead to frequent breakages of the nightly builds without anyone noticing. A better solution is to follow the LDC model on build the binaries as part of the CI systems on every PR. Hence, this project is about moving the release creation into dmd.

What are rough milestones of this project?

build and publish the Linux builds (tar archive)
build and publish the Linux DEB + RPM build
build and publish the Windows archive
build and publish the Windows installer
build and publish the OSX archive
build and publish the OSX dmg
build and publish FreeBSD archive
publish raw source as archive
code-sign Windows binaries
PGP-sign all releases

Optional extensions:

build official Docker image + setup auto-publish
integrate other third-party extensions (e.g. APT registry)

To be discussed:

whether to upload to GitHub releases or the current S3 bucket

How does this project help the D community?

It will allow to ship a 64-bit DMD executable on Windows. Moreover, it will eliminate the frequent nightly failures and ensure there are always built DMD binaries. Lastly, it will allow anyone in the DMD community to understand and improve the current release setup. Thus, allowing to integrate upcoming changes quickly.

Recommended skills

experience with Bash
experience with CI systems

Optional: experience with Docker

What can students expect to get out of doing this project?

The student will collect a lot of experience with building binary releases on all major platforms.
Moreover, he will learn more DevOps skills and gather experience on best-practices for reliable Continuous Deployment.

Point of Contact

@wilzbach @MartinNowak

References

https://github.com/dlang/installer
http://downloads.dlang.org/nightlies/
LDC CI configurations (e.g. https://github.com/ldc-developers/ldc/blob/master/azure-pipelines.yml)
https://dlang.org/gpg_keys.html

Related projects:

Language Server Protocol for D

A Language Server is meant to provide the language-specific smarts and
communicate with development tools over a protocol that enables
inter-process communication. The Language Server
Protocol (LSP)
defines the protocol used between an editor or IDE and a language server
that provides language features like auto complete, go to definition,
find all references etc.

Adding features like auto complete, go to definition, or documentation
on hover for a programming language takes significant effort.
Traditionally this work had to be repeated for each development tool, as
each tool provides different APIs for implementing the same feature (see
the list of Editors and IDEs).
Additionally as the existing tools use `libdparse` (used in
DLS and
DCD), they lack the ability to
perform semantic analysis on the AST and can't evaluate `mixin`'s
nor D's very powerful Compile-Time Function Evaluation (CTFE).

The idea behind the Language Server Protocol (LSP) is to standardize the
protocol for how such servers and development tools communicate. This
way, a single D Language Server can be re-used in multiple development
tools, which in turn can support multiple languages with minimal effort.

Recent development in the DMD compiler made it available as a DUB
library.
This mean that the compiler itself can be re-used and third-party tools
don't have to re-implement e.g. D's complex CTFE. This project is
about using the DMD compiler as a library to implement a Language Server
for D.

Proposed Project Mentors: @jacob-carlborg @wilzbach

std.serialization

D has numerous serialization frameworks. From std.json to
std.csv over protobuf to
Vibe.d's own serialization
framework.

The goal of this project is to implement a flexible
(de)serialization framework that can be used as a standardized building
block for various serialization related things. One important goal would
be to define how a user-specified type has to work in order to make it
serializable, as well as to allow the end-user (e.g. a user of std.json)
to change the serialization of third-party types that cannot themselves
be modified (i.e. let them define their own policies).

Starting points:

Point of Contact: @jacob-carlborg

Functional Reactive Programming

Functional reactive programming (FRP) is a programming paradigm for
reactive programming (asynchronous dataflow programming) using the
building blocks of functional programming (e.g. map, reduce, filter).
The best-known FRP API is the ReactiveX API whose
implementation in D this project would focus. Note that this project is
closely related to the std.eventloop project.

Unify CI system

Description

D has been using the auto-tester for a very long time (it was created in 2006 for D) and it has become a crucial part of D's infrastructure as it allows testing a PR on four platforms (Windows, Linux, OSX, FreeBSD) and 9 configurations (x86 <-> x86_64). However, as only one person has access to it, it is very hard for the community to make changes to it and integrate newer testing strategies which is why over the time many other CI systems have been added (Travis, CircleCi, SemaphoreCI, AppVeyor, Azure, Jenkins ...). Over the last year the entire project tester (the CI system which checks out the ~top50 dub projects and runs their test suite on every pull request) has been moved to Buildkite and we have made excellent experiences with it. In short, it allows to setup agents anywhere and connect to a central build server. This allows to use cheap or university resources, but also unavailable resources. For example, FreeBSD isn't available on most CI systems. As the aggregation of various CI systems leads to a lot of problems (frequent failures, no access, knowledge separation, upgrade issues), the goal of this project would be to unify all systems, s.t. all of them are run on Buildkite.

What are rough milestones of this project?

Migrate CircleCi to Buildkite
- CircleCi is used for CodeCov coverage report generation
Migrate SemaphoreCi to Buildkite
- SemaphoreCi is used to test building DMD with LDC, GDC and the latest DMD release
Migrate druntime AppVeyor
- AppVeyor is used to run the druntime projects on Windows
Migrate dmd Azure jobs
- Azure is used to test various Windows builds:
  - a DMD 64-bit compiler
  - building DMD with VisualD
  - building DMD with LDC
Migrate auto-tester builds

There are a few side dependencies:

build/auto-publish Debian docker image for Buildkite agents
setup agents for Windows (Step 3), OSX (Step 5) + FreeBSD (Step 5)

Extensions:

setup performance statistic agent
migrate from makefiles to D build files (#44)

How does this project help the D community?

A single CI system means that we can take full advantage of the individual features (e.g. Buildkite supports automatic retries of failing builds), no problems with CI machine access (you only need to be able to access one system, not 7), less confusion and missing knowledge (currently only very few people understand how all systems work).
Moreover, being a single system will make it a lot more reliable and easier to extend (e.g. DragonFlyBSD testing or Musl tests). However, most importantly we will be able to do upgrades and changes to the system a lot easier (e.g. upgrading from Debian 8 to 9, upgrade gcc, llvm, ...)

Recommended skills

Experience with Debian
Experience with CI systems
Experience with Bash
Experience with Docker

What can students expect to get out of doing this project?

As we currently use almost all popular CI services, the student will aggregate a lot of experience with them which very likely will help him for future jobs as well.
Moreover, he will learn more DevOps skills and gather experience on best-practices for reliable CI systems.

Point of Contact

@wilzbach, @MartinNowak

References

DWT/JPort

DWT is a library for building applications with a graphical user
interface. It's a port of the SWT Java library from Eclipse. It's a
full translation of the Java code to D. No Java code, JNI or JVM remains
in the DWT project. Currently Windows and Linux are the supported
platforms. So far the translation has only be done manually. There's
currently no support for macOS and DWT has not been updated to the
latest version of SWT.

For these reason a new project has been created, JPort, to automatically
translated Java code to D. It's based on JDT, the Eclipse Java
compiler. The idea is that it's not required to handle translating all
Java code, only what's necessary to translate SWT to D. This will
hopefully reduce the scope of the project. Even with this restriction
SWT is a very large code base and if JPort can successfully translate it
to D it would be able to handle other projects as well.

Here's the list of goals, not all goals need to be planned for:

Main goal : complete JPort and update DWT to the latest version of
SWT
Port DWT to macOS using JPort
Use JPort to translate JDT to D and JPort itself to D (it's
currently written in Scala)

Projects:

Point of Contact: @jacob-carlborg

Library for unix command line tools

Description

The aim of this project is to create a library that aids in the development of command line applications in unix systems, mainly providing a unified interface for:

parsing command line flags and arguments
parsing environment variables
parsing configuration files
accessing standard locations for configuration files and user data

There are many specifications for cli arguments such as:

Moreover common command line utilities adhere to freedesktop specifications for storing and accessing user data and configurations:

https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

There are already some experiments in this space, mainly:

Even if some of these libraries try to unify command line parsing and configuration file parsing, they either don't adhere to specifications or are not fully compliant.
Moreover some cli utilities could take into consideration environment variables

This library should provide a unified api to parse configuration files from standard locations, environment variables and cli arguments (in this order). Moreover the library could provide support to multiple file configurations format (sdl, yaml, toml, etc...) and standardized system locations.

What are rough milestones of this project?

To be discussed

How does this project help the D community?

This project helps the D community by providing an higher level api than std.getopt and fulfills the common needs of developers that build cli utilities for unix systems.

Recommended skills

Knowledge of the GNU/Linux operating system, shell and common cli utilities.

Point of Contact

@FraMecca

References

See above links

Find a way forward for std.experimental

Description

The packages in std.experimental are stuck in some limbo state.
We need to find a way forward for them.
And make sure not to end up in the same situation again.

What are rough milestones of this project?

find a way forward for the packages in experimental
move the packages along the way
kill experimental

How does this project help the D community?

It will reduce uncertainty about phobos

Recommended skills

grit

Point of Contact

the authers of the packages in experimental

References

Replace Runtime Hooks with Templates

D’s betterC mode is an important tool to be able to use D on bare-metal and embedded platforms. By disabling, for example the class support, the compiles does not need to as many runtime functions and types to be implemented for the code to compile and link. This makes the life of a bare-metal developer much easier.

But there is a catch! A lot of the language features runtime hooks, which in turn requires the TypeInfo class to be able to function. One way of solving both of these problems is by moving runtime hooks to use templates instead. This solves the betterC by removing the dependency of classes (the TypeInfo class), and it solves the safety issue because now the compiler will have all the information about the hook and it can verify it itself and not just trust that the runtime developer remembered to mark the hook correctly.

This proposal will work on translating all the array hooks from using the TypeInfo class to templates.

Point of contact: @RazvanN7

C++ interops

There are many experiments in transparently interfacing D with C++,
notably:

dpp by Atila Neves, that parses
and translates headers on the fly
Calypso by Elie Morisse that
bridges the two languages at the AST level.
DStep by Jacob Carlborg,
automatically generates bindings for C and Objective-C code. The
task would be to add support for C++.

There is also a lot of work in bringing STL types to the DRuntime (see
dlang/druntime#2310).

D would greatly benefit by seamlessly integrating the C++ ecosystem.
There are many ways to accomplish this goal and it should be discussed
with the community. Andrei already commented on the importance of a tool
like Calypso in this thread:
https://forum.dlang.org/post/[email protected]

Proposed Project Mentor: @andralex

Improve existing documentation on `vibe-core` asynchronous API

Description

The Vibe.d framework would benefit from improved documentation regarding the usage of its asynchronous API. This could be used to integrate the existing documentation and help new users familiarize with the low-level structure of the framework.

The current vibe-core repository lacks examples regarding how to use asynchronous I/O, while the main vibe.d repository contains a great number of application-level use cases.

Some of the missing use cases which could be demonstrated through examples are:

Basic Task usage: runTask, proper callback structure, exchanging messages between Tasks through send and receive primitives. Demonstrate Task execution flow control through yield().
WorkerTasks vs Tasks: an overview of the possible use cases of worker threads vs fibers (aka green threads, aka co-routines, aka Tasks). Usage of threads for heavy duty data processing by demonstrating an implementation of the actor model vs. using Tasks to implement e.g. an asynchronous request handler.
vibe.core.sync: Demonstrate usage of TaskMutex, RecursiveTaskMutex, TaskCondition while working with data shared between Tasks.
vibe.core.concurrency primitives: explain the logic behind Future and async and provide a simple usage example.

These examples could be included in the DLang Tour or displayed in a similar fashion on the vibe.d website and should be included in the vibe-core repository, example directory.

What are rough milestones of this project?

Task usage & WorkerTask vs Task examples
vibe.core.sync examples
vibe.core.concurrency examples
Inclusion of each examples in vibe-core (providing each example as a dub project)
Inclusion of the new vibe-core module documentation to the DLang Tour and / or vibed.org, possibly leveraging https://run.dlang.io to give the possibility of editing each example and rebuild it on the fly.

How does this project help the D community?

The Vibe.d web framework provides a simple yet extremely powerful interface for asynchronous I/O which can be exploited in a broad range of situations, such as web server implementation, concurrent data processing, the event-driven programming model.

Improving the existing documentation and providing more use cases in the form of examples would surely increase the interest in it and help new users familiarize with the vibe-core API, while allowing existing contributors to spot possible inconsistencies or bugs and fix them.

Recommended skills

Familiarity with the Vibe.d web framework.
Understanding of Vibe.d concurrency model and the difference between Threads, Tasks, the message passing interface (often referred to as the actor model).

Point of Contact

@GallaFrancesco

References

Recent feature request:
vibe-d/vibe-core#133

At the moment, the following documentation is available:

API:
http://vibed.org/api/vibe.core.core/
http://vibed.org/api/vibe.core.concurrency/

Features Description:
http://vibed.org/features

Examples:
https://github.com/vibe-d/vibe-core/tree/master/examples

Vibe.d in the Dlang Tour:
https://tour.dlang.org/tour/en/vibed/vibe-d-web-framework

Similar outcome (with sources)
https://tour.dlang.org/tour/mir
https://github.com/dlang-tour/mir

Actor model high-level description
https://en.wikipedia.org/wiki/Actor_model

Interface with C++ STL containers

Advance the ongoing effort to make C++ STL containers usable from D.
The project seed is here https://github.com/dlang-cpp-interop/stl-containers.

What are rough milestones of this project?

Provide a solution to hooking move semantics in D.
Fix DMD ABI and mangling issues which affect C++ interoperability
Finish the implementation of STL containers

Baremetal D runtime

Many features in the standard library are implemented by using the C
runtime. Reusing the rock-solid work of generations of programmers makes
a lot of sense as just outputting via `printf` can be very
complex.
However, for embedded environments a programmer often only has a tiny
amount of storage available or might not even have a C runtime available
for his target. The idea of this project is to implement the most basic
parts of the runtime natively in D, s.t. bare metals programmers can
just compile for their target. Implementing features natively in D also
means that other targets like Web
Assembly will
automatically be able to use them too. Another advantage of implementing
these basic building blocks in D is that one can take advantage of D's
Compile-Time Function Evaluation (CTFE) and optimize the methods even
further at compile-time.

Project focus: - Implement memcpy, memcmp and similar in D (with
similar performance to C) - Implement `malloc` and similar in D (with
similar performance to C)

Starting points: