GithubHelp home page GithubHelp logo

voxaai / voxa Goto Github PK

View Code? Open in Web Editor NEW
73.0 73.0 26.0 3.53 MB

Voxa is a framework that uses state machines to create elegant cross platform conversational experiences.

Home Page: http://voxa.readthedocs.io/

License: MIT License

JavaScript 1.31% Shell 0.12% Makefile 0.10% TypeScript 98.47%
alexa alexa-skills-kit nodejs sdk

voxa's People

Contributors

armonge avatar chrux avatar cvaca7 avatar dependabot[bot] avatar heneryville avatar lvelasquezm avatar mathmass avatar michael-krebs avatar omenocal avatar rmberrios avatar sloaisiga avatar snyk-bot avatar spatical avatar wuelcas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

voxa's Issues

Directive method requires three parameters when only needs one

Every directive has a writeToReply method that requires three parameters: reply, event and transition. Those three params are not used in all directives, most of the directives just use the first one.

When building a skill in vanilla Javascript you can just pass the one you need to pass, but when using Typescript you are obligated to pass the three params.

One practical example for using the writeToReply method inside a skill, is when writing the PlayAudio directive in a reply on a AudioPlayer.PlaybackNearlyFinished request. That directive just need the reply param, the other two params are not used at all.

voxaApp['onAudioPlayer.PlaybackNearlyFinished']((voxaEvent: IVoxaEvent, reply: AlexaReply) => {
  const playAudio = new PlayAudio({
    behavior: "REPLACE_ALL",
    offsetInMilliseconds: 0,
    token: "",
    url: 'https://domain.com/audio.mp3'
  });

  // If I pass only the reply param, Typescript will complain
  playAudio.writeToReply(reply, voxaEvent, {});

  return reply;
});

Separate flow control and Speech directives

Flow control and providing speech should be separated.

We'll make two core directives:

  • say will add speech to the reply
  • flow will determine what Voxa will do next. Values may be continue, yield or terminate. continue is the default behavior.

Add two meta-directives:

  • ask will append a say directive, and set the flow directive to yield
  • tell will append a say directive, and set the flow directive to terminate

I think that flow actually will be implemented less as a directive, and more as a special Voxa instruction, similar to to.

Create Changelog.md

We need to start using changelogs files to keep a record of our releases!

How to handle new RequestTypes?

As it is right now the ask-sdk from Amazon can easily handle new types of alexa requests without necesarilly running too deep into the Framework code. As an example, to support the new SessionResumedRequest for Skill Connections it's only needed to implement a simple request handler:

    canHandle(handlerInput) {
        const request = handlerInput.requestEnvelope.request;
        return request.type === 'SessionResumedRequest';
   }

Right now there's no documentation at all as to achieving something like that in Voxa, checking the code seems like we could extend the AlexaPlatform class with something like

class CustomAlexaPlatform extends AlexaPlatform {
  public getPlatformRequests() {
    return Array.prototype.concat(
        super().getPlatformRequests(), 
        'SessionResumedRequest'
    )
  }
}

We need to figure out if there's any changes required to support a "Better Way (tm)" or if this is sufficient. After that we need to document the accepted solution

Support for Dynamic Entities?

Hi team!

Has anyone explored using Alexa's dynamic entities within Voxa? I've been trying to get this to work, but Voxa doesn't seem to use my dynamic entities — despite them existing within directives throughout my use of the skill.

I've got a directive that has updateBehaviour set to "REPLACE" and a list of dynamic entity values as per these docs. I've managed to get it to work in a Node app without using Voxa, but Voxa doesn't seem to pick up the dynamic entities.

The custom directive concept I've been trying is summarised like so (I've got it broken up a bit more into a reusable directive that doesn't have the behavior and types hardcoded in below):

'use strict';

class DynamicEntityDirective {
  constructor(updateBehavior, types) {
    this.updateBehavior = updateBehavior;
    this.types = types;
  }

  async writeToReply(reply) {
    const response = reply.response;
    if (!response.directives) {
      response.directives = [];
    }

    reply.response.directives.push({
      type: 'Dialog.UpdateDynamicEntities',
      updateBehavior: 'REPLACE',
      types: [
        {
          name: 'dynamic-list',
          values: [
            {
              id: '1',
              name: {
                value: 'Dynamic Item 1',
                synonyms: ['Item 1', 'Item One']
              }
            },
            {
              id: '2',
              name: {
                value: 'Dynamic Item 2',
                synonyms: ['Item 2', 'Item Two']
              }
            },
            {
              id: '3',
              name: {
                value: 'Dynamic Item 3',
                synonyms: ['Item 3', 'Item Three']
              }
            }
          ]
        }
      ]
    });
  }
}

DynamicEntityDirective.key = 'alexaDynamicEntity';
DynamicEntityDirective.platform = 'alexa';

module.exports = DynamicEntityDirective;

Referred to in responses to initial launch intents like so:

return {
          directives: [buildDynamicEntityDirectives(voxaEvent)],
          reply: 'SampleResponse',
          to: 'entry',
          flow: 'yield',
};

Any ideas? Thanks for your help!

Voxa crashes with a poor error message if an unknown state is referenced

When I direct things to an undefined state, I get a crash with an error message that isn't very helpful. In the example below the unknown state is AddToList. I'd expect an error message about this not being a known state. Double points if you can propose states that it might have been, like git's command suggestions.

https://github.com/mediarain/voxa/blob/master/lib/StateMachine.js#L114

voxa onError TypeError: Cannot read property 'AddToList' of undefined +1ms
voxa TypeError: Cannot read property 'AddToList' of undefined
voxa at StateMachine.runCurrentState (/Users/mitchellharris/src/alexa/lists/node_modules/voxa/lib/StateMachine.js:114:34)
voxa at Promise.mapSeries.then (/Users/mitchellharris/src/alexa/lists/node_modules/voxa/lib/StateMachine.js:75:26)
voxa at tryCatcher (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/util.js:16:23)
voxa at Promise._settlePromiseFromHandler (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/promise.js:512:31)
voxa at Promise._settlePromise (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/promise.js:569:18)
voxa at Promise._settlePromiseCtx (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/promise.js:606:10)
voxa at Async._drainQueue (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/async.js:138:12)
voxa at Async._drainQueues (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/async.js:143:10)
voxa at Immediate.Async.drainQueues (/Users/mitchellharris/src/alexa/lists/node_modules/bluebird/js/release/async.js:17:14)
voxa at runCallback (timers.js:637:20)
voxa at tryOnImmediate (timers.js:610:5)
voxa at processImmediate [as _immediateCallback] (timers.js:582:5) +0ms

Lambda's Context Object is not being passed on

Hi,

I'm having an issue with Voxa library on AlexaSkill.js not passing on the context object that comes as a parameter for all Lambda handlers:

https://github.com/mediarain/voxa/blob/ec0d98d61f161970dcf52c6073aa42815b5ea2b3/lib/AlexaSkill.js#L93

We need to be able to change a property in the context item, specifically: context.callbackWaitsForEmptyEventLoop = false so that Lambda function will returns immediately as callback is called. Without this property, if we still have any nodejs event in the loop (e.g: shared resources connecting to redis/elasticCache, DB Connection), the function will just freeze and timed out (see: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html#nodejs-prog-model-context-methods).

Is it possible that this context item being exposed to the AlexaSkill object?

VOXA as a library is too big

The filesize of VOXA when included in another project has grown to be too large, currently when included in a project it is 19 MB. Can we make it so the docs aren't included? or how about the coverage folder?

I last tried with "voxa": "^3.1.0-alpha4",

/node_modules/voxa
4.0K LICENSE.md
4.0K Makefile
4.0K README.md
4.0K _config.yml
5.2M coverage
3.6M docs
124K hello-world
32K junit.xml
2.1M lib
684K node_modules
4.0K package.json
4.0K run-ci.sh
440K src
512K test
4.0K tsconfig.json
4.0K tslint.json
5.9M typedoc
80K yarn.lock

Voxa should give a better error when an intent is unhandled

Currently I get:
UnhandledState: Transition from entry resulted in null
at /Users/mitchellharris/src/alexa/alexa-podcasts/node_modules/voxa/lib/StateMachine.js:40:50
at tryCatcher (/Users/mitchellharris/src/alexa/alexa-podcasts/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/Users/mitchellharris/src/alexa/alexa-
etc

I'd expect something like, "XXX went unhandled"

Cloudwatch Plugin should report the absense of an error for statistical purposes

The CloudWatch plugin should send a 0 metric when a response causes no errors. This will help statistical measures. I think it'd be difficult to write some alarms against this metric w/o the 0 value.

Also, we should really think through the namespace for these cloudwatch events. "StateMachine" almost certainly doesn't make sense. That would cause all the skills running on the same environment to overwrite each other. We probably should use a dimension to differentiate between different skills. Speculatively, I wonder if we use just the right dimension and just the right namespace, we could get it to log under the Lambda function so that Caught Errors would appear right next to Errors and Invocations.

Finally. CloudWatch should expose on the request object a helper to let other spots in the skill log things to cloudwatch (e.g. successful order placements, or timedout/fake orders in Starbucks)

Errors in variables are masked

When an error in a variable occurs, it's nearly impossible to tell what it is. Often the error is just item[1] is not a function. We need better error messages with stack traces that point to the error itself, not to something in DefaultRenderer.

States and intents placed above the code that handle the entry state won't work

There's weird bug if I place code placed above the code that handle the entry state inside the register function in the states.js file. It's pretty simple to explain:

This code works great:

skill.onState('entry', {
  'AMAZON.CancelIntent': 'exitIntentState',
  'AMAZON.StopIntent': 'exitIntentState',
});

skill.onState('exitIntentState', () => ({ reply: 'ExitIntent.Msg' }));

skill.onIntent('LaunchIntent', () => ({ reply: 'Launch.Msg' }));

The LaunchIntent will not work in this scenario:

skill.onIntent('LaunchIntent', () => ({ reply: 'Launch.Msg' }));

skill.onState('entry', {
  'AMAZON.CancelIntent': 'exitIntentState',
  'AMAZON.StopIntent': 'exitIntentState',
});

skill.onState('exitIntentState', () => ({ reply: 'ExitIntent.Msg' }));

The SomeCustomIntent will not work in this scenario but the LaunchIntent will work just fine:

skill.onIntent('SomeCustomIntent', () => ({ reply: 'SomeCustomIntent.Msg' }));

skill.onState('entry', {
  'AMAZON.CancelIntent': 'exitIntentState',
  'AMAZON.StopIntent': 'exitIntentState',
});

skill.onState('exitIntentState', () => ({ reply: 'ExitIntent.Msg' }));

skill.onIntent('LaunchIntent', () => ({ reply: 'Launch.Msg' }));

If you test the states/intents that are placed above the entry state, the skill will return An unrecoverable error occurred

Clean Voxa installation with typescript is broken

Starting a new voxa project results in in the following when running tsc

[10:21:26 AM] Starting compilation in watch mode...

node_modules/voxa/lib/src/VoxaApp.d.ts:2:8 - error TS1259: Module '"/home/armonge/workspace/old-skills/categoriesgame/node_modules/i18next/index"' can only be default-imported using the 'esModuleInterop' flag

2 import i18next from "i18next";
         ~~~~~~~

  node_modules/i18next/index.d.ts:986:1
    986 export = i18next;
        ~~~~~~~~~~~~~~~~~
    This module is declared with using 'export =', and can only be used with a default import when using the 'esModuleInterop' flag.

node_modules/voxa/lib/src/VoxaEvent.d.ts:3:10 - error TS2305: Module '"../../../i18next"' has no exported member 'default'.

3 import { default as i18n } from "i18next";
           ~~~~~~~

node_modules/voxa/lib/src/renderers/Renderer.d.ts:1:8 - error TS1259: Module '"/home/armonge/workspace/old-skills/categoriesgame/node_modules/i18next/index"' can only be default-imported using the 'esModuleInterop' flag

1 import i18n from "i18next";
         ~~~~

  node_modules/i18next/index.d.ts:986:1
    986 export = i18next;
        ~~~~~~~~~~~~~~~~~
    This module is declared with using 'export =', and can only be used with a default import when using the 'esModuleInterop' flag.

[10:21:33 AM] Found 3 errors. Watching for file changes.

From what i can see that is a result of this recent change in i18next https://github.com/i18next/i18next/pull/1291/files

Basically the references to i18n have to be updated

Session attributes are aggresively persistant

Our handling of session attributes is make it practically impossible to remove stuff from the session. In StateMachineSkill line 53 (the onBeforeReplySent handler), the old session, and the result of the data serialize are merged together.

I'm mostly OK with the idea that you've got to delete things explicitly from the session to make them not persist, (though my functional part of me hates this), but that data values from prior request are allowed to clober the next request is not OK. Voxa should enforce that everything the model wants to go to the next session will be in serialize, and it should not merge in the prior data attributes. This is why the original statemachine code used Object.assign here, rather than _.merge.

States with an intents array filter should have priority over the ones that don't

Suppose you have this code:

voxaApp.onState("myExampleState", { ... });
voxaApp.onState("myExampleState", { ... }, "YesIntent");

If the user triggers the YesIntent and the state in the session is myExampleState, the code that will run is the first controller that Voxa founds of that state, in this case is the one without the "YesIntent" filter. For this case, if you don't have a code that handles the YesIntent it may land in a fallback or unhandle state, and that leads to frustration.

The order on how we structure our state controllers matter in Voxa and that's great, but in this kind of situation I think Voxa should prioritize the controllers with an intent filter over the ones that don't even if the controller without an intent filter is the first one in the code structure, just because how explicit setting that filter is.

A seasoned Voxa dev can know that controllers with intent filters goes on top of the one that don't, but someone that's getting around Voxa things can't tell that, and it can be frustrating to know why I can't land on the controller that I'm being explicit about, and it's more frustrating that Voxa doesn't throw a clue on why that happens.

RFC #1: Directives

Voxa RFC #1: Directives

Background:

All voice-first devices allow for alternative non-verbal channels of communication.
Some allow for control of non-skill functionality, such as a long-form audio player, or sign in.
Some also allow for delegation of skill responsibility to the Alexa system (such as Alexa's delegate directive).
See Appendix A for a list of non-verbal commands that Alexa, Google Home, and Cortana support.

Voxa needs a way to support these non-verbal directives, such that:

  1. Some directives contain end-user facing text (e.g. Alexa display templates). This text should be I18N compatible, and specified in the view.
  2. Some directives are control flow based. (e.g. an account linking card, or Dialog.Delegate) and should be specified from the controller, with no mention in the view.
  3. Directives should be extensible to new platforms, and new platform capabilities
  4. Directives should full advantage of the complete range of capabilities of each platform, that is, they should not be restrained to the lowest-common-denominator of all platforms.
  5. Directives should be easy to specify compactly in the controller, and not require extensive platform switching.

Terminology

  • Transition - A is a JS object returned from a controller that specifies transition instructions, which define how a skill will handle a given state.
    This is intended to be as light-weight as possible to make reading controllers easy.
    The following example returns a transition that has two transition instructions.
    	skill.onIntent('NextIntent', event => {
    		return { to: '?confirm-next', say: 'Navigation.ConfirmNext'}
    	})
  • Composed Transition - Represents a set of final decisions for each state. Voxa will use the composed transition to determine how to handle a state. A composed transition is created by a transition through a process called composition. The composed transition carries the following facts:
    • nextState: The state to which Voxa will flow to next, if it is non-yielding, or will resume at if it is yielding
    • reply: The platform-specific reply object.
    • flow: One of the following three strings: next, yield, or halt. In the case of next Voxa will flow to the next state without sending a response, yield will send send a response to the user, and allow the user to respond. halt will send a final response to the user, and halt the conversation.

Directive Objects

This RFP introduces the notion of a directive.
Directives are appended to the directives transition instruction to specify and parameterize non-verbal responses.
A directive may be used as follows:

return {
	to: 'halt',
	say: 'Playback.StartOver',
	directives: [ Alexa.PlayAudio(episode.url, episode.id, 0) ]
}

Here, the Alexa.PlayAudio directive instructs Alexa to play a given episode URL, with the token id, and to start at 0 ms.

Each Platform module (e.g. require('voxa').Alexa) will provide at it's root level directive functions that define all directives supported on that platform.

Directive functions are appended to the response in the directives transition instruction. Each directive will be composed by Voxa in the order it's defined.
Composition is carried out by invoking the built directive (as a function call) with the reply object, and the voxa event object.
Example below:

(url, token, offsetInMilliseconds) => new
 (alexaReply, voxaEvent) =>  
 	if (voxaEvent.platform != 'alexa') return;
	alexaReply.response.directives.push({
		type: "AudioPlayer.Play",
		playBehavior: "REPLACE",
		audioItem: { stream: { token, url, offsetInMilliseconds }}
	})

Responsiblities of a directive

A directive must:

  • Silently pass on platforms that it is unable to affect. e.g. the PlayAudio directive in the above example will only operate on Alexa events
  • Call reply.yield() if the directive causes a yield to the user (e.g. the Dialog.Delegate directive on Alexa)
  • Modify the reply object such that it's specific operation is usable by the platform

Trade offs

This approach makes several trade-offs:

  1. It is platform specific. This is to allow each directive to be maximally relevant to it's target platform, and avoid the lowest-common-denominator problem of a more generic approach.
  2. It is a bit verbose since all directives are in the directives section. We anticipate adding (later) the ability to specify directives as key/value pairs

Platform specific shortcuts

The transition object should be as short and reable as possible.
Invoking directive builders and appending to a directives array in the transition is a bit verbose.
As an alternative recommendation, platforms will have specific shortcuts for well-known directives, such as playing or cards.
For example, defined below is a set of directives that apply exlusively to the Alexa platform.

return {
	to: 'halt',
	say: 'Playback.StartOver',
	alexa: {
		play: {url: episode.url, token: episode.id, offset: 0},
		card: 'cards.order-receipt'
	}
}

The Alexa platform adapter will register an onTransitionChanged handler that will map these declarative directive definitions to the directives array form, where they will be used normally.
It is expected that Cortana, Google Home, and any other platforms would use a similar process.

Speech as a directive

The say command is really a directive as well.
It should be modified to be written as one.

Changes necessary

  • Rename relevant objects in Voxa to follow the new names defined in terminology. Namely, transition, transition instruction, compilation, and compiledTransition.
  • Add new hooks to Voxa for pre and post compiled transitions
  • Rename voxa/lib/adapters to voxa/lib/platforms
  • Re-write the reply object. There should be one reply object for each platform. Creation is handled by the platform. The reply object is very specific to the platform it's targetting, except for some common definitions (e.g. for speech) so that plugins can have a predictable idea of what's being said.
  • An easy way for directives to access rendering. Support access to this on a skill object, that is present on the voxaEvent object. e.g. voxaEvent.skill.views.render. This will allow the directives an easy path to something that's able to render in I18N their attributes.
  • Rework the platforms to do their final replies on composed objects.

Alexa Audio Directives

Usage:

The response to a request to start the current episode over.

return {
	to: 'halt',
	say: 'Playback.StartOver',
	directives: [ Alexa.PlayAudio(episode.url, episode.id, 0) ]
}

Implementation:

(url, token, offsetInMilliseconds) => (alexaReply, voxaEvent) =>  
 	if (voxaEvent.platform != 'alexa') return;
	alexaReply.response.directives.push({
		type: "AudioPlayer.Play",
		playBehavior: "REPLACE",
		audioItem: { stream: { token, url, offsetInMilliseconds }}
	})

Alexa Dialog Directive

Usage:

A reponse to a request to order an airplane ticket

return {
	to: "book-flight",
	directives: [ Alexa.DialogDelegate({origin: 'Seattle'}) ]
}

Implementation

(slots) => (alexaReply, voxaEvent) => {
	alexaReply.yield();
	alexaReply.response.directives.push({
		type: "Dialog.Delegate",
		updatedIntent: {
			name: voxaEvent.intent.name
			confirmationStatus: ...,
			slots: _.mapValues( (v,k) = {name: k, value: v}
		}
	})
}

Display Templates

Usage:

Imagine showing a recipe

const views = {
	"templates": {
		"recipe-details": {
			type: "BodyTemplate1",
			backButton: "VISIBLE",
			title: "{recipeTitle}",
			textContent: { ... }
		}
    }
};

return {
	to: "?recipe-details",
	say: "Recipe.Details",
	directives: [ Alexa.RenderTemplate('templates.recipe-details', `recipe-${recipe.id}`) ]
}

Implementation

(templatePath, token) => alexaReply, voxaEvent => {
	if(_.isString(templatePath)) alexaReply.directives.push(_.merge(voxaEvent.skill.views.render(templatePath),{token}))
	else alexaReply.directives.push(voxaEvent.skill.render(templatePath))
}

Alexa Home Account Linking

Usage:

return {
	to: 'halt',
	say: 'Error.NotAuthorized',
	directives: [ Alexa.AccountLinkingCard() ]
}

Implementation

  () => (alexaReply, voxaEvent) => {
  	alexaReply.card = {type: "LinkAccount"}
  }

Cortana Sign-in Card

Usage:

return {
	to: 'halt',
	say: 'Error.NotAuthorized',
	directives: [ Cortana.SignInCard() ]
}

Implementation

() => (cortanaReply, voxaEvent) => {
	let card = new botbuilder.SiginCard();
	card.text(...)
	cortanaReply.attachments.push(card);
}

Dialog Flow Basic Card

Usage:

Example is showing a recipt via a card after an order

const views = {
	cards: {
		"order-receipt": {
			title: "Your Order",
			text: "Your order was {order} and cost {pricing}",
			button: {text: 'Print', url: '{receiptUrl'},
			image: {url: 'http://example.com/image', alt: 'Logo', display: 'CROPPED'}
		}
	}
}

return {
	to: 'halt',
	say: 'Order.Success',
	directives: [ DialogFlow.BasicCard('cards.order-receipt') ]
}

Implementation

(templatePath) => dialogFlowReply, voxaEvent => {
	const cardDesc = await voxaEvent.skill.views.render(templatePath).
	const card = dialgFlowReply.action.addBasicCard(cardDesc.text)
	_.forEach(cardDesc, (key,value) => {
		switch(key) {
			case 'title': card.setTitle(value); break;
			case 'text': break;
			case 'button': card.addButton(value.text, varlu.url) break;
			case ...: ...
			default: throw new Error('Unknown basic card description...')
		}
	})
}

Appendix A

  • Alexa Directives
    • Audio directives
    • Dialog directives
    • Display directive
    • Hint directive
    • Home Card
    • Account Linking card
  • Cortana Attachments
    • Cards
    • Sign-in Card
    • Suggestions
  • Dialog Flow Responses
    • Basic Card
    • List Selector
    • Carousel Selector
    • Suggestion Chip
    • Sign in prompt
    • Transactions

Request flow diagram does not show onBeforeStateChanged correctly

The documentation shows that onBeforeStateChanged executes only after the first state.
screen shot 2017-03-24 at 9 47 56 am

However, it runs on all states. It should instead be shows as

onBeforeStateChanged => runTransition => onAfterStateChanged

Also, it's difficult to tell from the flow diagram what is a hook and what is logic. Maybe we could use colors to describe this.

Finally, the name onBeforeStateChanged isn't really correct, it's before the state is entered. Maybe call it onStateWillExecute, or onStateEnter, and the other should be onStateExit, or onStateExecuted.

Ask directive points to an object

In order to make the response editor in Command Center more efficient, we'll need to have the Ask directive work with an object, rather than an individual string. The object should be:

{
   "ssml": "This is what will be said",
   "reprompt": "This is the reprompt"
}

The key that holds the content should accept any of the following:

  • ssml
  • ask
  • content
  • say

Link doc with the actual example if exists

I was reading the documentation on github and I was wondering if the doc about account linking, for example, has a sample project, it would be great to connect them.

state-flow plugin crashes when doing fallback

The state-flow plugin crashes when there is no fallback. It fails here:

  skill.onAfterStateChanged((alexaEvent, reply, transition) => {
    alexaEvent.flow.push(transition.to);
  });

Should onAfterStateChanged be called if we're in fallback? If so, we need to put loud notes in docs that transition could be null. However, we've already got onUnhandledState, so I'm not sure this is necessary.

Serialize responses to JSON before returning control flow.

Voxa builds it's response in a custom Response object that is then serialized via the .toJSON() function. However, voxa does not actually call this function, instead allowing the system to call it (lambda or express). This make it very confusing to know with which state of the object you're working. Recently virtual-alexa will not properly update it's state unless the object is itself a valid Alexa response POJO, so we should have Voxa call .toJSON() on it's own.

They way voxa handles different types of request is really confusing!

Currently we handle different types of AlexaRequest.

const AlexaRequests = [
  "AudioPlayer.PlaybackStarted",
  "AudioPlayer.PlaybackFinished",
  "AudioPlayer.PlaybackNearlyFinished",
  "AudioPlayer.PlaybackStopped",
  "AudioPlayer.PlaybackFailed",
  "System.ExceptionEncountered",
  "PlaybackController.NextCommandIssued",
  "PlaybackController.PauseCommandIssued",
  "PlaybackController.PlayCommandIssued",
  "PlaybackController.PreviousCommandIssued",
  "AlexaSkillEvent.ProactiveSubscriptionChanged",
  "AlexaSkillEvent.SkillAccountLinked",
  "AlexaSkillEvent.SkillEnabled",
  "AlexaSkillEvent.SkillDisabled",
  "AlexaSkillEvent.SkillPermissionAccepted",
  "AlexaSkillEvent.SkillPermissionChanged",
  "AlexaHouseholdListEvent.ItemsCreated",
  "AlexaHouseholdListEvent.ItemsUpdated",
  "AlexaHouseholdListEvent.ItemsDeleted",
  "Connections.Response",
  "Display.ElementSelected",
  "CanFulfillIntentRequest",
  "GameEngine.InputHandlerEvent",
  "Alexa.Presentation.APL.UserEvent",
  "Messaging.MessageReceived",
];

but in order to use it, you need to know which are marked as on${whateverRequestType} or be treat as an intent like CanFulFillIntentRequest.

My proposal to fix this issue is to treat all the other request as onStates except(OnrequestStarted, OnsessionEvents, etc. - lifecycle events). This change will also
fixes #242
Because there won't be a need to manually include new event request types. It doesn't matter the event, Voxa will just work!.

Thoughts? If you like my proposal give a thumbs up or down! You can also propose a new approach!

100 MB in dependencies is way too much!

Is there a way we can reduce the number of dependencies?
actions-on-google has 49MB and googleapis has 44 MB.

This issue is really troublesome due to lambda size limit of 250 MB

The reply instruction should permit multiple values

I should be able to return multiple view paths for the reply attribute in state handlers. This will allow re-using responses easier. This should be possible:

skill.onState(request => ({reply: ['A','B']}))

If the non-last elements are a tell or an ask, an exception should be thrown.

Add AlexaFlush directive

Add an Alexa directive called FlushToProgressResponse that will kick off sending all of the speech gathered so far to the Alexa progressive response API, and clearing out the speech. This should replace Alexa using the say directive for progressive responses.

Some requirements:

  • This directive should do nothing if there is no speech built up yet. It should not initiate any network requests in this situation.
  • Any speech built up in the current transition should be added in BEFORE this directive runs. That is, this directive should execute with lower priority than the other say, ask, tell directives in the same transition.
  • We can think about changing the name. However, it'll be used infrequently enough that a long descriptive name is probably OK.

Access user's location (Google Action)

We're currently working on implementing a PermissionsRequest in order to determine a departure location. We would like to be able to ask permission to the user to access his location.

Reply and Directive re-work

In order to reduce the coupling between replies and directives, we'll re-work replies to be more specific to their platform, and directives to own their own mutation logic.

Things that aren't working on in the existing system

  • In Alexa, the directive sets the card object on the reply, but the reply has to know to pick that up for the output
  • Same for alexa say
  • Same for reprompt in Alexa
  • In Google Home, knowledge about how to add a suggestion is in the DialogFlowAdapter rather than the directive
  • All of this is an indication that directives and replies are over-coupled

Reply become as close as possible to their underlying platform representation

The first step in this change is to re-write replies. Instead of replies having a generic representation, then being serializable to their platform's representation, we'll have the reply actually hold their platform's representation in them. That means that the AlexaReply will look like a partially built Alexa response, and the DialogFlowReply will hold the RichResponse as a core member variable, rather than just in it's render step.

This means that well clear out the VoxaReply object of any common base-class implementation, since there will no longer be an intermediate representation between the reply and it's actual end representation to the platform. The purpose of this is to make it easier for directives to be able to write their effects into the reply. The VoxaReply base class will now become an interface. However, since speech is so general across all the platforms, the interface will have operations that allow generic access to speech, mostly for use by plugins.

Directive modifications

Directives need to do two things:

  1. Build themselves based on their paramters
  2. Write themselves to the underlying platform's reply

To do this, directives will be represented as a function, that returns an implementor of the IDirective interface. The implementors of the class will specify as a member variable the platform that this directive applies to. Directive rendering will ignore directives not intended for their platform. Again, "core" will apply to all platforms.

The IDirective will have a single method on it called writeToReply(voxaReply, transition) that will allow the directive to either modify the voxaReply, or add new directives to the transition if it is a meta-directive. This is where the DialogFlow.Suggestion directive would call voxaReply.richResponse.addSuggesion.

Dialogflow editor response has no filfillmentText when using v.3.3.0

I've recently been experimenting with the newer versions of Voxa and Dialogflow, but I'm finding that in versions of Voxa after the change from DialogFlowPlatform to GoogleAssistantPlatform, whenever I test it out in Dialogflow, the response is "Not Available" — even though it does seem to send it through to Dialogflow in the response.

In quick summary, the fulfillmentText and fulfillmentMessages when using tell in a views.json model don't seem to come through. Only a richResponse?

For example, on a simple LaunchIntent:

image

If I look into the diagnostic info from Dialogflow though, I do see:

"richResponse": {
  "items": [
    {
      "simpleResponse": {
        "textToSpeech": "<speak>Welcome!</speak>"
      }
    }
  ]
},

image

The code I'm using is pretty simple and largely based on the hello-world example.

app.onIntent('LaunchIntent', (voxaEvent) => {
  const youSaid = _.get(voxaEvent, 'rawEvent.queryResult.queryText');
  console.log('Saw: ', youSaid);
  return {
    reply: 'LaunchIntent'
  };
});

When the response is given via sayp or textp is seems to work though:

app.onUnhandledState(async (voxaEvent) => {
  const youSaid = _.get(voxaEvent, 'rawEvent.queryResult.queryText');
  return {
    sayp: 'You said ' + youSaid,
    textp: 'You said ' + youSaid,
    flow: "terminate"
  };
});

image

"fulfillmentText": "You said unhandled huh?",
"fulfillmentMessages": [
  {
    "text": {
      "text": [
        "You said unhandled huh?"
      ]
    }
  }
],

Is this by design? Is there a new way to ensure Dialogflow also sees the messages and gets fulfillmentText and such?

Thank you for your help!

Error on SessionEndedRequest is ignored by Voxa

If the skill receives an improper directive, audio exceeding time limit, etc. It may try to end the session. if the reason is an Error We should throw an error in the Handler

alexaEvent
...
"request": {
        "type":"SessionEndedRequest",
        ...
        "locale":"en-US",
        "reason":"ERROR",
        "error": {
            "type":"INTERNAL_SERVICE_ERROR",
            "message":"Internal Server Error || The total duration of audio content exceeds the maximum allowed duration"
        }
    }

Type 'Lambda Log | Lambda Log' is not assignable to type 'Lambda Log'.

./run-ci.sh

  • NODE_ENV=development
  • yarn install --frozen-lockfile
    yarn install v1.17.3
    warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
    [1/4] 🔍 Resolving packages...
    success Already up-to-date.
    ✨ Done in 0.66s.
  • NODE_ENV=development
  • yarn run lint
    yarn run v1.17.3
    $ tslint --config tslint.json --project tsconfig.json
    ✨ Done in 5.51s.
  • NODE_ENV=development
  • yarn run test-ci
    yarn run v1.17.3
    $ NODE_ENV=local.example nyc mocha --reporter mocha-jenkins-reporter test test/.spec. test/**/.spec.

/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:226
return new TSError(diagnosticText, diagnosticCodes)
^
TSError: ⨯ Unable to compile TypeScript:
src/services/Storage.ts:18:5 - error TS2322: Type 'LambdaLog | LambdaLog' is not assignable to type 'LambdaLog'.
Type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/voxa/node_modules/@types/lambda-log/index").LambdaLog' is not assignable to type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/@types/lambda-log/index").LambdaLog'.
Types of property 'options' are incompatible.
Type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/voxa/node_modules/@types/lambda-log/index").LambdaLogOptions' is not assignable to type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/@types/lambda-log/index").LambdaLogOptions'.
Types of property 'dynamicMeta' are incompatible.
Type '((message: import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/voxa/node_modules/@types/lambda-log/index").LogMessage) => any) | undefined' is not assignable to type '((message: import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/@types/lambda-log/index").LogMessage) => any) | undefined'.
Type '(message: import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/voxa/node_modules/@types/lambda-log/index").LogMessage) => any' is not assignable to type '(message: import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/@types/lambda-log/index").LogMessage) => any'.
Types of parameters 'message' and 'message' are incompatible.
Type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/@types/lambda-log/index").LogMessage' is not assignable to type 'import("/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/voxa/node_modules/@types/lambda-log/index").LogMessage'.
Types of property 'toJSON' are incompatible.
Type '(format?: number | undefined) => string' is not assignable to type '(format?: boolean | undefined) => string'.
Types of parameters 'format' and 'format' are incompatible.
Type 'boolean | undefined' is not assignable to type 'number | undefined'.
Type 'false' is not assignable to type 'number | undefined'.

18 this.log = event ? event.log : new LambdaLog();
~~~~~~~~

at createTSError (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:226:12)
at getOutput (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:335:40)
at Object.compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:368:11)
at Module.m._compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:414:43)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4
at Object.require.extensions.(anonymous function) [as .ts] (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:417:12)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/src/services/User.ts:5:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Module.m._compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:414:23)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4
at Object.require.extensions.(anonymous function) [as .ts] (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:417:12)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/src/app/index.ts:20:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Module.m._compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:414:23)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4
at Object.require.extensions.(anonymous function) [as .ts] (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:417:12)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/test/tear-up-helpers.ts:13:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Module.m._compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:414:23)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4
at Object.require.extensions.(anonymous function) [as .ts] (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:417:12)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/test/alexa/launch.spec.ts:9:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Module.m._compile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:414:23)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4
at Object.require.extensions.(anonymous function) [as .ts] (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/ts-node/src/index.ts:417:12)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at /Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/mocha/lib/mocha.js:250:27
at Array.forEach (<anonymous>)
at Mocha.loadFiles (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/mocha/lib/mocha.js:247:14)
at Mocha.run (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/mocha/lib/mocha.js:576:10)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/mocha/bin/_mocha:637:18)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Module.replacementCompile (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
at runMain (/Users/rahul.patwa/.node-spawn-wrap-93339-984d468c4e43/node:68:10)
at Function.<anonymous> (/Users/rahul.patwa/.node-spawn-wrap-93339-984d468c4e43/node:171:5)
at Object.<anonymous> (/Users/rahul.patwa/voxa-starterkit-qa-isobar/node_modules/nyc/bin/wrap.js:23:4)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
at /Users/rahul.patwa/.node-spawn-wrap-93339-984d468c4e43/node:178:8
at Object.<anonymous> (/Users/rahul.patwa/.node-spawn-wrap-93339-984d468c4e43/node:181:3)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)

----------|----------|----------|----------|----------|-------------------|

File % Stmts % Branch % Funcs % Lines Uncovered Line #s
All files 92.86 75 0 100
index.js 92.86 75 0 100 7
---------- ---------- ---------- ---------- ---------- -------------------
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.