liplum / calcupiano

A multiplatform musical calculator powered by Flutter.

How to generate Liplum

prompts = """
1 girl,green hair,deep green hoodie,deep green hoodie cap,
deep green clothes,plum,plum hairpin,plum-shaped decoration,
deep red eyes,white face,white neck,eyelashes,
blushed,Minecraft zombie,medium breasts
"""

# You can generate your own Liplum.
liplum = model(prompts)
liplum.activate().join(WORLD)

while True:
  liplum.working()
liplum.die() # no, please no😢

calcupiano's Issues

A soundpack editor and preview

Derived from #1 .

Basic Support

• Manage soundpacks: adding, removing, renaming, etc.
• A SoundpackMeta editor.
• Change preview.png.
• Duplicate a soundpack.
• Preview a soundpack.
• Export a soundpack.
• Support preview.png.

Advanced Support

• L10n editor.
• Manage its sound files.

A contribution guide

Basic Support

• How to build CalcuPiano.
• Code style.
• Localization.
• Soundpack making.
• Sheet music making.

Re-implement the serialization with migration and polymorphism.

Proposal

The current way requires lots of handy work and doesn't integrate with the dart:convert which is native supported.

Since the JsonCodec in Dart allows custom toJson(toEncodable) and fromJson(reviver) and functions well.

Tasks

• Use custom toEncodable and reviver to support migration and polymorphism.
• Test toJson.
• Test fromJson.
• Test migration.
• Test polymorphism.

Sheet Music

Proposal

1. Users cannot memorize all the sheet music with numbered notation.
2. Users like sharing their works and playing new songs from the community.

Basic Support

• Allow users to compose a song by recording what keys they tapped.
• Automatically playing a sheet music.

Advanced Support

• Play multiple soundtracks simultaneously.
• Dynamically change the soundpack when playing song.
• New file format of sheet music.

Accessibility Design

Proposal

CalcuPiano is based on sound and touch. It's not limited for the visually impaired and even the hearing impaired to play with.

Basic Supports

• More Semantics for Screen Reader.
• Physcial feedback if the visually impaired have pressed a wrong key in the sheet music.
• Visual feedback on the screen to represent a note.

Interactive Feedback

Basic Supports

• Play animations when user taps the Piano Key.

Andvanced Supports

• Vibrate the phone with the same rythm if user taps the Piano Key.

Author, email and URL of built-in soundpack

Proposal

Contributors may add more built-in soundpacks. Signature and acknowledgement are necessary.

Basic Support

• The metadata of built-in soundpacks.
• Easy to open the URL and email. Temporarily cancelled.

Advanced Support

• User can check the metadata of all kinds of soundpacks with the same experience.

Play sound depend on the pressure of touch

Basic Support

• The different pressure of touch will change the pitch(?) or volumn(?).
• A toggle to turn it on/off.

Worth Mentioning

• Some platforms don't support pressure detecting, such as web. So hide the toggle in this situation.

More Reading

1. ForcePressGestureRecognizer.
2. Pressure property in PointerEvent.

As to the second particle, if the platform or the pressure can't be detected, the pressure will be 1.0 as default.

Customizable Sound Pack

Basic Support

• Change current sound pack.
• Import custom sound pack.
• #7
• Implement the structure of soundpack.

Advanced Support

• Manage soundpacks: adding, removing, renaming, etc.
• Duplicate a soundpack.
• #13
• Preview a soundpack.

Worth Mentioning

• Display the sound packs in alphabetical order? Or addition time? Or the frenquency user used?
• How to save the indices of sound packs?

Audio cache doesn't work on Safari that causes latency

When I started a simple http server with python. I typed plus key several times with Safari on my iPad.
It also happens to macOS Safari.

E:\MyProject\CalcuPiano\calcupiano\build\web>python -m http.server 80
Serving HTTP on :: port 80 (http://[::]:80/) ...
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:25] "GET / HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:26] "GET /manifest.json HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:26] "GET /flutter.js HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:26] "GET /main.dart.js HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:26] "GET /assets/FontManifest.json HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:26] "GET /version.json?cachebuster=1671981626887 HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/1.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/l10n/en.json HTTP/1.1" 304 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/2.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/1.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/3.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/2.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/4.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/3.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/5.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/4.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/6.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/5.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/7.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/6.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/8.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/7.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/9.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/8.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/9.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/minus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:27] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/mul.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/minus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/div.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/mul.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/eq.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/div.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:28] "GET /assets/assets/soundpack/default/eq.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:30] "GET /assets/assets/soundpack/default/9.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:30] "GET /assets/assets/soundpack/default/5.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:30] "GET /assets/assets/soundpack/default/mul.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:30] "GET /assets/assets/soundpack/default/5.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:30] "GET /assets/assets/soundpack/default/mul.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:31] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:31] "GET /assets/assets/soundpack/default/minus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:31] "GET /assets/assets/soundpack/default/minus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:32] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:32] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:32] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:32] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:32] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:33] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:33] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:33] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:33] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -
::ffff:192.168.1.9 - - [25/Dec/2022 23:20:33] "GET /assets/assets/soundpack/default/plus.wav HTTP/1.1" 200 -

Independent attributes for each soundpack and keyboard

Proposal

Since CalcuPiano will support multiple keyboard, see #15, users may apply different soundpacks or attributes to different keyboards.
Meanwhile, Soundpacks might vary in volume, pitch or play rate. Users may want to adjust them to a comfortable level.

Basic Support

• #18
• Users can multiplicatively adjust them again for each keyboard.

Local audio files don't work on iOS

When I duplicate a soundpack, and try to play any key of it, the Flutter console outputs
Unexpected platform error: AVPlayerItem.Status.failed

Cache sounds before users really tap down

Proposal

On Calcupiano Web, audio files need to be sent and cached locally. That will cause latency of feedback between tapping and sound playing.
User may feel confused when first time play Calcupiano or just after refreshing the page.
On the physical devices, it's not a real problem because reading local files is fast enough.

Basic Support

• Cache the sounds currently used on the first screen.
• Cache the sounds once users switch to a new one.

Allow adjustment of volume, pitch, play rate for each soundpack.

Implementation

Save those attributes into soundpack itself.
More specifically, save the adjustment into metadata, soundpack.json.

Proposal

1. This allows users to publish it with good adjustment.
2. Remeber the states after serialization.

Support of split screen, aka, limited screen space

Proposal

Many devices, for example, XiaoMi products, support split screen.
Sometimes users will split the screen, one for sheet music, and one for keyboard, as what a real piano player does XD.

Basic Supports

• Only display the Piano Keyboard if there is no enough room for the screen, for it's easy to tap the key.
• Hide other buttons or make an all-in-one menu button to navigate or change settings related to playing quickly.
• Don't do hardcoded padding or whatever based on magic numbers, because we can't imagine what devices users could even play with.

Advanced Supports

• Overlay Piano Keyboard. Is this possible with only flutter? Or the heavy platform specific jobs should be taken? Or only work on Android?

Multiple Keyboards (Two-Hand Mode in Landscape)

Proposal

Users might play the ensemble with both hands.
This fits perfectly with landscape mode.

Basic Support

• Replace the Sheet Screen with another Keyboard.
• Allow users to choose different Soundpack for each keyboard.

Advanced Support

• Multiple keyboards if the screen has enough room, e.g., using tablet.

Add an `About` page

Proposal

CalcuPiano now has contributors, it's essential to introduce them.

Basic Support

• Including translators
• Including programmers

The file format of a soundpack.

A soundpack:

• It should have audios of each note.
• It could have some built-in sheet music.
• A preview.
• An icon.
• The metadata, including, author, description, url...
• Its own l10n, what has been mentioned in #5.

Stop the sound when tap up

Basic Support

• Instantly stop the sound if user taps up.
• A toggle to turn it on/off.

Details

Use a key to remeber the audio player in TapDown event, and stop the sound in TapUp event.

Exchange two SoundFiles in Soundpack Composer

Proposal

It's easy to fix wrong sound file position of notes by exchanging two SoundFiles in Soundpack Composer.

Basic Support

• Exchange two files in the same soundpack when edited.
• Drag and drop two files in the same soundpack that's being edited.

Internationalization

Basic Supports

• Internationalization of CalcuPiano's UI.
• English support. Is this a real task?

Worth Mentioning

• Dynamically l10n to avoid the switch hell. Meanwhile, the advanced supports are likely involved.

Advanced Supports:

• Switch language.

What's Switch Hell

If you have ever worked with the built-in localization, you must meet this:

enum ShpaeType {
  circle, rectangle, triangle, ellipse, trapezoid, rhombus, pentagon, hexagon, octagon;
}

Now you have to do the l10n with:

enum ShapeType {
  // The values...
  String localized(BuildContext context){
    switch(this){
      case ShapeType.circle:
        return context.localization.circle;
      // Repeat this for each enum value...
    }
  }
}

Physical Keyboard Support

Proposal

Although users have the best experience with touchscreen, desktop users or someone who uses external physical keyboard still matter.

Basic Support

• Support the numpad.
• Allow key mapping.
• Close the virtual keyboard on the screen.
• Remeber the states last used, window size, whether closed vitrual keyboard...

Advanced Support

• Bind a virtual keyboard with a physical keyboard, so the ripple effect will display.

DONT USE CONTROLLER TO PLAY CALCUPIANO