Interface features for handheld devices about flare-engine

[clintbellanger]

For menus, I think we can use the Widget base class to add interaction without mouse/pointer/touch.

Think of how in an operating system you can use Tab to highlight buttons/links/etc in order. I think we can use the current keybindings for up/down/left/right to navigate menus and the accept keybinding for clicking or dragging/dropping.

Perhaps we need some kind of Widget List class that contains all the widgets in a menu. If we know the positions of each widget in our list, we should be able to interpret the directional keypresses to navigate and highlight (Focus) those widgets in the right order.

This may mean that Inventory Slots, Power Slots, and Action Bar Slots need to be re-created as Widgets. Perhaps the Widget base class can have an activate(), drag(), and drop() that can be implemented to handle generic interactions between Widgets.

This approach (making everything Widgets and operating over collections of Widgets) is much preferred than trying to implement arrowkey movement a different way for each menu (might not be feasible anyway, as all the menus can be configured with different layouts).

Not all widgets should allow Focus. Example, many labels in the game don't need to be part of the "tab order".

I think if a widget has Focus, we can display the tooltip for that widget. That's how we can handle Power and Item tooltips without "mouseover".

[dorkster]

I like the idea of a WidgetList class.

The mouse pointer still exists, even if it isn't being used. How about we use the directional keypresses to snap the mouse cursor to each widget? Then two face buttons could replicate the behavior of mouse buttons (although dragging with a joypad might need to be like #121).

This means we would keep our existing activate/drag/drop logic for icon slots. They'd still need to become Widgets so they could be added to the WidgetList.

[clintbellanger]

Ah, sounds like an interesting idea. I like it. We should double check that it's possible with a hardware mouse pointer (e.g. try SDL_WarpMouse).

[dorkster]

I've started a rough implementation of this here: dorkster@48553af. SDL_WarpMouse did the trick.

[clintbellanger]

I have some thoughts about these issues.

Some of them are great and sensible -- I've opened a new ticket to add additional features to the Accept key (talk to NPC, open chest, pick up loot, etc).

I'm concerned about tacking on virtual mouse movements to get full keyboard support. Ever play a game that was an obvious lazy console port (or the other direction) and the clunky interface showed? Flare's menu system is obviously designed around mouse usage. If I were creating a gamepad-only or keyboard-only game I would design the menus very differently.

I'm reducing our priority on this until we have a better solution. Full keyboard menu support won't be a must-have feature before we enter Beta.

I really believe that the Correct way to do this would be to create a full new UI with gamepad/keyboard in mind, and merge that implementation in. Maybe it would involve changing menus to some compromise layout that works for both pointer and pointerless systems. Maybe it would involve making e.g. an abstract base MenuInventory and several implementations, which can be chosen via engine config file.

I know we would really love having full keyboard/gamepad support for Flare, as it would hit many more devices. But we must deliver a quality solution. And a proper solution is going to take a ton of work. Maybe we can experiment with this in a branch during Beta. Or we can make a second game that has mainly gamepad play in mind, and refactor the menu system to flex and handle both games.

[dorkster]

The recently implemented mouse emulation, while not optimal, allows for joystick only gameplay. It should bu suitable until we implement a joystick-specific interface.

However, there's one thing left that still requires a keyboard. The name entry textbox in GameStateNew starts empty. We should set a DEFAULT_NAME for Flare (something like "Adventurer"). Setting the DEFAULT_NAME variable "locks" that textbox, so that behavior should be removed here in the engine.

[clintbellanger]

One default name would technically work, but it would make the loading screen a bit less useful -- each game slot would contain the same hero name. Perhaps instead we can give a unique default name per portrait? And it displays unless the player has typed anything into the name field already.

[dorkster]

@clintbellanger That sounds like a much better idea and allows for a lot of flexibility. We could define the names for each portrait in hero_options.txt.

[clintbellanger]

I'll post some names for the current portraits in a bit

[clintbellanger]

I added default names for flare-game. Perhaps someone can double-check them for any obvious screw-ups. E.g. a name that is always masculine given to a female, or a fantasy name that means something filthy in some language.

A few of the names are plays on the actual people portrayed in the portraits. I can change these or add more of these if we want.

I suppose we want to allow these to be translated/localized? These names are probably okay if the player reads the latin alphabet, but e.g. completely different names would be better for a slavic language. We can add logic to xgettext.py to gather these into the data.pot file.

[igorko]

I didn't find branch with last pull request by @josephbleau and this implementation looks good for me, so I recovered his work in my branch. See https://github.com/igorko/flare-engine/tree/keyboard_input

[igorko]

Please look here if this is right way to forbit hero moving
igorko@b740b17
It works by blocking keypresses, but atm I don't have an idea how to relocate keypress for menu navigation needs.

[clintbellanger]

igorko, the standard Flare way to do this is use input locks on the arrow keys. If the menus are active and the player presses any arrow key, it should "lock" that arrow key. Then the hero movement should check (if it does not already) to make sure the arrow isn't locked before moving.

Input key locks basically mean that although a key is in the "pressing" state, some function is already using that key press. The menu logic is run before the gameplay logic each frame so that the higher level menus can use and lock input before it reaches the lower layers. That's what this comment means in GameStatePlay.cpp:

// check menus first (top layer gets mouse click priority)

Although we mostly apply these locks for mouse clicks, we can just as easily lock[UP] = true, etc.

Side note for clarity, key locks also let us have keypresses register once for an action. Otherwise holding the button will repeat the action, instead of requiring the player to release the button and press again. Sometimes we actually want to allow holding the button, which is e.g. why we don't lock a key when attacking -- we want the player to be able to hold a key to continually attack.

[igorko]

Here is some work at WidgetScrollBox at https://github.com/igorko/flare-engine/tree/scrollbox I have tried to add scrolling support to WidgetScrollBox, but got some wrong behavior. Buttons are not activated if I store their pointers in vector<Widget*>. Seems like derived activate() is not called. Also scrolling doesn't work very good.

[igorko]

@dorkster Thanks for feedback and help!

[igorko]

Here is my attempt to add keyboard support to MenuManager https://github.com/igorko/flare-engine/tree/keyboard. It is not I want to see there, but it works. If anyone has ideas to make it better(don't use inpt and inpt->mouse for getting slot position using keyboard...), please give feedback.

[dorkster]

@igorko Now that MenuItemStorage isn't using inpt directly, I've continued what you started here: https://github.com/dorkster/flare-engine/tree/keyboard

What works now:

• Rearranging/equipping inventory items
• Moving items between inventory and stash
• Buying and selling items. Since the vendor menu can't be reorganized by the player, keyboard dragging isn't used here as with the stash. Instead, a single press of ACCEPT behaves like holding CTRL + SHIFT when using the mouse. Much faster than having to "drag" items between the menus.
• Moving inventory items with powers (like potions) to the action bar

What doesn't work (yet):

• Rearranging powers on the actionbar
• Dropping items from the inventory

In addition, I did my best to fix a few bugs that were lying around, such as resetting keyboard dragging if the mouse is clicked.

I acknowledge that an ideal implementation of this feature would probably involve abstracting the current logic so that the same code (more or less) can be used for both mouse and mouse-less controls. However, I think it's more important to get an implementation working, even if that means separate code that contains some duplication.

[igorko]

After a bit of testing I discovered next problems:

1. After trading with vendor, closing menus and opening inventory, it doesn't have focus. You can move hero with arrow keys instead.
2. It would be nice to use two clicks for equipping item instead of navigating to destination.
3. Dunno why but I don't see magenta frame after selecting item inside inventory.(As my idea) that should indicate slot selection on first click. Second click activates item(e.g. potion) and clears magenta selection.The same with equippment: if you click second time on the same slot - you equipp item, but if you click on another slot - you move item to that slot. In that time source slot is highlighted with magenta.
4. I can't use HP Potion inside inventory. Look how I implemented item activating in my initial commit. It was done with two clicks.
5. If I select item inside inventory and activate ActionBar (or it's already activated - read "slot != -1") floating icon of selected item is moved to ActionBar. I still can move item inside inventory but it's a little bit confusing.

[dorkster]

1. You're right. And I found another focus bug, too. I'll look at those.
2. I figured single clicking would be faster, but I guess it's better for item activation. I'll re-implement it.
3. same as 2
4. same as 2
5. Yeah, it's a bit strange. Same with the powers menu. Not sure what to do about that yet (maybe defocus?)

[dorkster]

Alright, I put 2-stage clicking back in. If the slot is checked (magenta border), it will activate, buy, sell, or stash the item depending on context. If it has a blue border, it will either reorganize the item or return it to its previous slot, depending on context.

I also seemed to have fixed the focus bugs in the process.

[igorko]

It works nice. I see only two problems now:

1. point 4 from previous comment(about floating icon). We could use something like "menu focus" as it's done for stash-inventory, though ActionBar is always active. I think it will be next solution: last key press(arrows or z/x moves "selection" to corresponding menu(actionbar or inventory) and switches floating icon to that menu)
2. we should somehow handle slot selection by mistake. If I select potion and don't want to use it anymore the only one solution is to move it to different slot. We could use ESC to deselect dragging item. ESC is used for Main Menu navigation and Game Exit Menu so anyway it's "must have" key.

[igorko]

In result we get next buttons for handheld:
4 arrow keys
Select(Enter)
Cancel(Escape)
2 arrow keys for ActionBar
ActionBar Select key

so 4 buttons (joystick) + 5 buttons

and we have at least one free button(afaik such device should have at least 2+4 buttons + joystick)

[dorkster]

I think it will be next solution: last key press(arrows or z/x moves "selection" to corresponding menu(actionbar or inventory) and switches floating icon to that menu)

I think that's the best solution there. That way, we'd able to start the floating icon on its source menu, even if the actionbar is already selected.

We could use ESC to deselect dragging item. ESC is used for Main Menu navigation and Game Exit Menu so anyway it's "must have" key.

That sounds good to me. Taking it one step further, the CANCEL key should probably have 2-3 stages in this order: cancel dragging, defocus selected tablists (maybe do this with cancel dragging), show the exit menu.

[dorkster]

I've added the above changes to my branch.

I playtested for a a bit with a controller and things feel pretty good. However, I noticed that MenuNPCActions doesn't have keyboard/controller support yet, so I'll try to add that.

[dorkster]

I've implemented mouse-less interaction for MenuNPCActions, so that's done.

I've also done some more playtesting. I've found it annoying that you need to use 2 clicks to use a power. It's really awkward in combat. Perhaps that extra button we have can be used as a shortcut for activating the highlighted power.

[igorko]

@dorkster, using two clicks was proposed by @clintbellanger . Let's wait for his feedback.

[igorko]

How I see this: the proper way to replace two-step clicking with two buttons is to remake WidgetSlot::checkClick() and not remaking MenuManager. BUT we have two different WidgetSlot "instances" : one for general usage which uses arrows and ACCEPT and second for ActionBar which uses z/x and ACTIONBAR. So remaking WidgetSlot to use two buttons for select/accept will require two extra buttons for two instances(we can't use the same extra button because ActionBar is always active and should work in parallel to other menus.) We could trace slot change as it already should be done by @dorkster(didn't test) and use one ACTIVATE extra button within all menus(too complex? If no - I'm ok with it). And dunno what case will be easier for end user.

[dorkster]

Here's my idea. I think it's simple and efficient.

We make sure the actionbar always has a slot selected and prevent defocusing it. The ACTIONBAR key is only used for trigger the menu shortcuts and picking up/dropping powers. We then use MAIN1 for triggering the currently selected power.

[igorko]

MAIN1? I don't understand what you mean. It's mouse button(by default). But if you see it clear, you can do it. I just was telling what problems I see.

[dorkster]

Maybe not MAIN1, as that might cause conflicts, but one of the keys that's used on the actionbar normally (like 1 or MAIN2).

[stefanbeller]

I'll highlight @zear to have him here to :)

I dislike the double ACTIONBAR key to use a power now, actually if you'd put MAIN1 to a key instead of a mouse, I'd imagine the proposal of @dorkster will work out nicely:

• ACTIONBAR is not overloaded with double tapping
• MAIN1 is also not overloaded, we just need to make sure, to tell people it can be a keyboard key.

Here is another proposal:
Once you selected the power in the power menu, the icon pops up as a highlight, now you need to use ACCEPT or CANCEL (automatic cancel on arrow keys as well) to move it to the actionbar:

• Select power/item with arrow keys
• use ACTION_BAR_BACK/FORWARD to select slot
• Press ACCEPT to highlight power in power menu,
• Press ACCEPT again to actually move the power into the action bar slot

This will leave only one option at the ACTIONBAR key, which is activating the action from the selected slot.
Using the ACCEPT button multiple times feels natural as well (think of deleting a save slot: accept on delete button and accept again on the safety question)

[dorkster]

I implemented my idea, substituting MAIN1 for BAR_1 and it works great. It made adding support for rearranging actionbar items simple as well.

I also fixed the inpt->mouse hijacking when keyboard dragging. This allows use to rearrange actionbar items while the player is moving around.

Lastly, I added joystick bindings to keybindings.txt. Since there's no GUI elements for remapping joystick buttons yet, you'll have to update your keybindings.txt by either:

A) Reseting to default settings via the config menu
or
B) Deleting your local keybindings.txt

[igorko]

Now I see how it works. Looks cool. But using BAR_1 is little bit confusing. I would like to see another new keybinding (we have one free) set to e.g. N keyboard button by default(it doesn't really matter. It's easy to refactor) and not mix BAR hotkeys with mouseless navigation keys. Because if ActionBar has focus on it, I can't use BAR hotkeys anymore, only BAR_1 for focused slot. Enabled mouseless support should not break common implementation, only extend it.

[dorkster]

Ah, I thought it was okay to override normal functionality there. I've gone ahead and added an ACTIONBAR_USE key, which is by default assigned to N on the keyboard.

[igorko]

Changing this to discussion and removing milestone.

[igorko]

This was a long time ago already and implemented, so closing issue.