azure / azure-functions-eventgrid-extension

EventGrid extension for Azure Functions

Event Grid Extension for Azure Functions

This extension provides functionality for receiving Event Grid webhook calls in Azure Functions, allowing you to easily write functions that respond to any event published to Event Grid.

Branch	Status
dev

License

This project is under the benevolent umbrella of the .NET Foundation and is licensed under the MIT License

Contributing

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-functions-eventgrid-extension's Issues

Support strongly typed event data

Currently EventGridEvent is designed with its Data property as a JObject which will force everyone downstream to get into dealing with converting that into their own strong types using the JSON.NET APIs themselves.

I propose supporting an EventGridEvent<T> where the Data property would now be T. This would allow people to use a parameter such as EventGridEvent<CustomerProfileUpdated> and it would "just work". Simultaneously we can keep the existing, default EventGridEvent by simply changing it to be a subclass of EventGridEvent<JObject> that effectively locks it into JObject for people who might choose to work with that model instead.

Is there a way to configure EventGridTrigger locally to use hybrid relay endpoint?

Hello,

I managed to configure console app to be triggered locally via hybrid connection, but I can't find anything in documentation regarding EventGridTrigger. Am I missing something?

Error getting SystemKey for v2 function app

I try to get the systemKey for v2 function app following instructions on https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-grid#version-2x-runtime-2 , using address: http://{functionappname}.azurewebsites.net/admin/host/systemkeys/eventgrid_extension?code={masterkey}. It returns 404 Not Found.

I also created a v1 function app to test. The v1 url works. However the v1 url does not work.

Event Grid Trigger missing direction in function.json

For a C# function using the Event Grid trigger, the generated function.json (/bin/Debug/netcoreapp3.1/MyTrigger/function.json) is missing the "direction" attribute.

According to the documentation, the "direction" attribute with a value of "in" is required for an EventGridTrigger attribute.

The function.json file appears as follows:

{
  "generatedBy": "Microsoft.NET.Sdk.Functions-3.0.3",
  "configurationSource": "attributes",
  "bindings": [
    {
      "type": "eventGridTrigger",
      "name": "eventGridEvent"
    }
  ],
  "disabled": false,
  "scriptFile": "../bin/myfunction.dll",
  "entryPoint": "Company.Function.EventGridTriggerCSharp1.Run"
}

Environment

• VS Code: 1.47.0-insiders
• .NET Core: 3.1.301
• Azure CLI: 2.8.0
• Azure Functions Core Tools: 3.0.2630

The functionName query parameter is always "raw" member name vs "logical name" supplied by FunctionNameAttribute

Because of this line of code:

azure-functions-eventgrid-extension/src/EventGridExtension/EventGridTriggerAttributeBindingProvider.cs

Line 49 in 9a9b3ee

return Task.FromResult<ITriggerBinding>(new EventGridTriggerBinding(context.Parameter, _extensionConfigProvider, context.Parameter.Member.Name));

The functionName querystring parameter for the web hook will always be the "raw" member name of the .NET class rather than the "logical name" that might be assigned by the FunctionNameAttribute.

I'm not sure how big of an issue this ultimately is, but as I was testing locally I expected to put the logical name (like you would for an HTTP triggered function) and it wasn't working and that's how I found this bug.

Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridEvent is obsolete, but new version doesn't work with EventGridTrigger

I'm attempting to use EventGridTrigger with EventGridEvent, but the EventGridEvent referenced in the docs is marked obsolete. It says to use the one from Microsoft.Azure.EventGrid.Models. But when I do that, I get the following error:
Microsoft.Azure.WebJobs.Extensions.EventGrid: Can't bind EventGridTriggerAttribute to type 'Microsoft.Azure.EventGrid.Models.EventGridEvent'.

Is there a new EventGridTrigger in a different package that works with this EventGridEvent?

I'm using functions v1 and running it as .net 461.

Extension does not await batch execution and can cause deadlocks/failed deliveries

We recently had a number of failed deliveries and the following Azure alert which seems to suggest the extensions is to blame:

Alert

We’ve detected that threads are mixing synchronous code with asynchronous code

Azure App Service has identified that 125 threads may be mixing synchronous code with asynchronous code. Application code is calling Task. Result and is synchronously waiting on the Task object, which can lead to deadlocks in the process.
The function calling Task.Result is Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridExtensionConfigProvider+d__9.MoveNext(), but the entire thread's call stack should be reviewed to identify the blocking pattern. Please review the call stack closely and make sure that your code is using the C# await keyword.

GCFrame
HelperMethodFrame_1OBJ
System.Threading.ManualResetEventSlim.Wait
System.Threading.Tasks.Task.SpinThenBlockingWait
System.Threading.Tasks.Task.InternalWaitCore
System.Threading.Tasks.Task`1[[System.__Canon System.Private.CoreLib]].GetResultCore
Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridExtensionConfigProvider+d__9.MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridExtensionConfigProvider+d__9 Microsoft.Azure.WebJobs.Extensions.EventGrid]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridExtensionConfigProvider+d__9 Microsoft.Azure.WebJobs.Extensions.EventGrid]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][Microsoft.Azure.WebJobs.Extensions.EventGrid.EventGridExtensionConfigProvider+d__9 Microsoft.Azure.WebJobs.Extensions.EventGrid]].MoveNext
System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction
System.Threading.Tasks.Task.RunContinuations
System.Threading.Tasks.Task.FinishContinuations
System.Threading.Tasks.Task`1[[System.__Canon System.Private.CoreLib]].TrySetResult
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon System.Private.CoreLib]].SetResult
System.Net.Http.HttpContent+d__70`2[[System.__Canon System.Private.CoreLib][System.__Canon System.Private.CoreLib]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][System.Net.Http.HttpContent+d__70`2[[System.__Canon System.Private.CoreLib][System.__Canon System.Private.CoreLib]] System.Net.Http]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][System.Net.Http.HttpContent+d__70`2[[System.__Canon System.Private.CoreLib][System.__Canon System.Private.CoreLib]] System.Net.Http]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.__Canon System.Private.CoreLib][System.Net.Http.HttpContent+d__70`2[[System.__Canon System.Private.CoreLib][System.__Canon System.Private.CoreLib]] System.Net.Http]].MoveNext
System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction
System.Threading.Tasks.Task.RunContinuations
System.Threading.Tasks.Task.FinishContinuations
System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib]].TrySetResult
System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult
System.Net.Http.HttpContent+d__54.MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.HttpContent+d__54 System.Net.Http]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.HttpContent+d__54 System.Net.Http]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.HttpContent+d__54 System.Net.Http]].MoveNext
System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction
System.Threading.Tasks.Task.RunContinuations
System.Threading.Tasks.Task.FinishContinuations
System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib]].TrySetResult
System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult
System.Net.Http.StreamToStreamCopy+<g__DisposeSourceAsync
0_0>d.MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.StreamToStreamCopy+<g__DisposeSourceAsync
0_0>d System.Net.Http]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.StreamToStreamCopy+<g__DisposeSourceAsync
0_0>d System.Net.Http]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.Net.Http.StreamToStreamCopy+<g__DisposeSourceAsync
0_0>d System.Net.Http]].MoveNext
System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction
System.Threading.Tasks.Task.RunContinuations
System.Threading.Tasks.Task.FinishContinuations
System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib]].TrySetResult
System.IO.Stream+d__30.MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.IO.Stream+d__30 System.Private.CoreLib]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.IO.Stream+d__30 System.Private.CoreLib]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult System.Private.CoreLib][System.IO.Stream+d__30 System.Private.CoreLib]].MoveNext
System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction
System.Threading.Tasks.Task.RunContinuations
System.Threading.Tasks.Task.FinishContinuations
System.Threading.Tasks.Task`1[[System.Int32 System.Private.CoreLib]].TrySetResult
System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1[[System.Int32 System.Private.CoreLib]].SetResult
Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream+d__11.MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32 System.Private.CoreLib][Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream+d__11 Microsoft.AspNetCore.Server.IIS]].ExecutionContextCallback
System.Threading.ExecutionContext.RunInternal
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32 System.Private.CoreLib][Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream+d__11 Microsoft.AspNetCore.Server.IIS]].MoveNext
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Int32 System.Private.CoreLib][Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream+d__11 Microsoft.AspNetCore.Server.IIS]].MoveNext

Scenario

Our binding looks like this:
public async Task ProcessStorageEventsAsync([EventGridTrigger] EventGridEvent[] eventGridEvents) { ... }

Potential cause

I believe the issue is that when a notification is received and the binding is to an array (as ours is) then the EventGridExtensionConfigProvider goes down the "Batch Dispatch" route in which the bound function execution is not awaited, as it is in "Single Dispatch" here.

Suggested fix

Move the await Task.WhenAll(executions); call outside of the if (_listeners[functionName].SingleDispatch) {} else {} statement

EventGridTrigger function registration

I was able to use event grid trigger before with no issue but now I'm not sure why I can no longer register a subscriber.

Project file:
<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <TargetFramework>netstandard2.0</TargetFramework> <AzureFunctionsVersion>v2</AzureFunctionsVersion> </PropertyGroup> <ItemGroup> <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.EventGrid" Version="2.0.0" /> <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.22" /> </ItemGroup> <ItemGroup> <None Update="host.json"> <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> </None> <None Update="local.settings.json"> <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> <CopyToPublishDirectory>Never</CopyToPublishDirectory> </None> </ItemGroup> </Project>
Code:
`
using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Microsoft.Azure.WebJobs.Extensions.EventGrid;
using Newtonsoft.Json.Linq;

namespace FunctionApp3
{
public static class Function1
{
[FunctionName("FuncNetStandard")]
public static void Run([EventGridTrigger]JObject req, ILogger log)
{
log.LogInformation("C# Event Grid trigger function processed a request.");

    }
}

}
`

Release dates?

I don't understand:

1. Which version should I use?
2. Why all versions are (still) in beta?
3. When will it be production-ready?
4. Is there a road-map?
5. How can the engine be not in beta (https://azure.microsoft.com/en-gb/blog/announcing-the-general-availability-of-azure-event-grid/) but the client is still in beta? Is there an SDK I am missing (sorry if this is the case, I am just trying to use NuGet)?

Thank you.

We need to support the newer cloud event

https://docs.microsoft.com/en-us/azure/event-grid/cloudevents-schema#cloudevent-schema

Input binding for Event Grid does not work for out of proc languages

For input binding,

1. if we are using batch type, it is expected that the parameter type be a JArray. For out of proc languages. the parameter type would usually be a string. This ends us throwing an exception. We should also check if the requested type is string and treat the binding accordingly.
2. if we are using a single event type, we expect the parameter to be a string or JObject. Even if it is a string, we parse it to a JObject. This breaks scenarios where users are passing a JArray of events. When triggered from the portal, you are very highly to hit this scenario causing all sorts of errors. We should try to parse it as a JArray as well.

TLDR; we expect binding type to be a JSON object. That's not a fair expectation, we should also allow JSON Arrays.

Fix code issues

In reviewing a recent PR, I saw a bunch of stuff which should be fixed:

• many files missing file headers
• EventGridListener should not be public. Some other types are public as well which should probably not be (e.g. SubscriptionValidationResponse)
• EventGridTriggerAttribute should be in the Microsoft.Azure.WebJobs namespace. See all other binding attributes for comparison. This would be a breaking change for v1, but we should fix for v2
• In the v2.x branch, the file EventGridEvent contains 2 types that seem completely unrelated. Are those types even used anymore?

Strong name the dev branch assemblies

Details and prior work:
Azure/azure-webjobs-sdk#1883

EventGrid Trigger for Functions does not work with EventTriggerAttribute in VS Tooling

When using the sample from here: https://docs.microsoft.com/en-us/azure/event-grid/resize-images-on-storage-blob-upload-event
everything works as expected - the event from a blog storage blob upload is fired.

However, when creating a new Azure Function from scratch in VS (not a .csx. but a compiled one) and add a reference to Microsoft.Azure.WebJobs.Extensions.EventGrid (1.0.0.-beta2) it compiles correctly, but after publishing to Functions in Azure, nothing happens when I add a file to the blob storage container.

No error, nothing.

I can see and verify that the subscription is in place - via:
az eventgrid resource event-subscription list --resource-name --provider-namespace Microsoft.Storage --resource-type storageAccounts

Any ideas what could be wrong?

Thanks!

EventGrid Trigger function returns Error: connect ECONNREFUSED 127.0.0.1:7071 when debugged locally

When I try to debug an EventGrid Trigger function locally, it returns Error: connect ECONNREFUSED 127.0.0.1:7071.
Request and response details are as given below.

POST http://localhost:7071/runtime/webhooks/EventGrid?functionName=UserWishlistEventSubscriber
Error: connect ECONNREFUSED 127.0.0.1:7071
Request Headers
aeg-event-type: Notification
Content-Type: application/json
User-Agent: PostmanRuntime/7.29.2
Accept: /
Postman-Token: b7c95cde-90dd-49da-93b5-10dd51016b6d
Host: localhost:7071
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Request Body
[
{
"id": "'1",
"eventType": "yourEventType",
"eventTime":"10:59:00.000",
"subject": "yoursubject",
"data": { "FirstName": "John", "Surname": "Smith" },
"dataVersion": "1.0"
}
]

What is going wrong here?

Event Grid Performance

I'm using event grid in our current project. I have created multiple topic endpoints for each event type and we ran out for space. Now I would like to reuse the existing topics so I am planning for below two options.

One topic for few event types (Or)
Only one topic for all event types.

I have below questions/confirmation.
Performance affect If I go with the second approach so Should I go and have one topic for few events or Should I keep only one topic.

Event Grid trigger always returns HTTP 202

From @glaserbeamer on April 20, 2018 2:38

Summary:
When an Event Grid trigger function throws an exception, I would expect an HTTP 500 to be returned to Event Grid; instead, this returns an HTTP 202, so Event Grid believes that processing was successful, and it never replays the event.

Repro steps:

1. Add new "Event Grid trigger" function (C#).
2. Change body of function to "throw new Exception();"
3. Save and run.

Sample log output (using default/sample event):
2018-04-20T02:21:43.875 [Info] Function started (Id=)
2018-04-20T02:21:44.125 [Error] Exception while executing function: Functions.EventGridTriggerCSharp1. mscorlib: Exception has been thrown by the target of an invocation. f-EventGridTriggerCSharp1__1818170464: Exception of type 'System.Exception' was thrown.
2018-04-20T02:21:44.187 [Error] Function completed (Failure, Id=, Duration=313ms)

HTTP return code:
202 Accepted

Copied from original issue: Azure/azure-functions-templates#673

Does not support Newtonsoft.json 9.0.1+

{
  "frameworks": {
    "net46": {
      "dependencies": {
        "Microsoft.Azure.Devices": "1.5.1"
      }
    }
  }
}

with above project.json which restore Newtonsoft.json 10.0.3, we will get a

[Error] Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexingException : Error indexing method 'Functions.EventGrid-CSharp' ---> System.InvalidOperationException : Can't bind EventGridTriggerAttribute to type 'Newtonsoft.Json.Linq.JObject'.

as a workaround, add an explicit reference to Newtonsoft.json 9.0.1:

"dependencies": {
    "Microsoft.Azure.Devices": "1.5.1",
    "Newtonsoft.json 9.0.1"
}

The function runtime crashes when using Microsoft.Azure.WebJobs.Extensions.EventGrid 2.0.0-beta4

Today I was updating my Nuget Packages and when updating the Microsoft.Azure.WebJobs.Extensions.EventGrid to 2.0.0-beta4 (currently using beta2) the following error occurs:

[30-8-2018 13:28:41] A ScriptHost error has occurred
[30-8-2018 13:28:41] System.Private.CoreLib: Could not load type 'Microsoft.Azure.WebJobs.Hosting.IWebJobsStartup' from assembly 'Microsoft.Azure.WebJobs.Host,Version=3.0.0.0, Culture=neutral, PublicKeyToken=null'.

The same is also when using beta3.
I am running on Functions V2.

The binding type(s) 'eventGridTrigger' are not registered. Please ensure the type is correct and the binding extension is installed.

I have two projects using the AzFunc runtime v. 1.0.19 and newest tooling in VS (15.10.2009.0). Both are referencing Microsoft.Azure.WebJobs.Extensions.EventGrid (2.0.0-beta4) and ExtensionsMetadataGenerator (1.0.0). When I run one of the project I get the following error:

The following 1 functions are in error:
[05-09-2018 07:31:21] CatalogUpdated: The binding type(s) 'eventGridTrigger' are not registered. Please ensure the type is correct and the binding extension is installed.

When comparing the two projects, they seem identical in references and types, so I am clueless as to why one would work when the other doesn't. How would I go about debugging this?

Why does EventGridTriggerBinding need to support multiple types to convert from?

Hi all, I was attempting to rebase PR #12 on top of the latest changes and ran into an interesting problem. A unit test was written for the EvenGridTriggerBinding::BindAsync method here:

azure-functions-eventgrid-extension/test/Extension.tests/UnitTests.cs

Lines 20 to 44 in 9a9b3ee

	public async Task bindAsyncTest()
	{
	MethodBase methodbase = this.GetType().GetMethod("DummyMethod", BindingFlags.NonPublic | BindingFlags.Instance);
	ParameterInfo[] arrayParam = methodbase.GetParameters();
	ITriggerBinding binding = new EventGridTriggerBinding(arrayParam[0], null, null);
	// given GventGridEvent
	EventGridEvent eve = JsonConvert.DeserializeObject<EventGridEvent>(FakeData.singleEvent);
	JObject data = eve.Data;
	ITriggerData triggerDataWithEvent = await binding.BindAsync(eve, null);
	Assert.Equal(data, triggerDataWithEvent.BindingData["data"]);
	ITriggerData triggerDataWithString = await binding.BindAsync(FakeData.singleEvent, null);
	Assert.Equal(data, triggerDataWithString.BindingData["data"]);
	// test invalid, batch of events
	FormatException formatException = await Assert.ThrowsAsync<FormatException>(() => binding.BindAsync(FakeData.arrayOfOneEvent, null));
	Assert.Equal($"Unable to parse {FakeData.arrayOfOneEvent} to {typeof(EventGridEvent)}", formatException.Message);
	// test invalid, random object
	var testObject = new TestClass();
	InvalidOperationException invalidException = await Assert.ThrowsAsync<InvalidOperationException>(() => binding.BindAsync(testObject, null));
	Assert.Equal($"Unable to bind {testObject} to type {arrayParam[0].ParameterType}", invalidException.Message);
	}

This unit test now passes in two totally different types of source values which the "old" EventGridTriggerBinding did support. They are string and EventGridEvent.

The question I have is: why does this binding need to support more than one? The binding itself is an internal class that closely cooperates with EventGridExtensionConfig's IAsyncConverter<HttpRequestMessage, HttpResponseMessage> whose responsibility it is to provide the value into the binding... albeit indirectly. Seen here:

azure-functions-eventgrid-extension/src/EventGridExtension/EventGridExtensionConfig.cs

Lines 85 to 96 in 9a9b3ee

	string jsonArray = await req.Content.ReadAsStringAsync();
	List<EventGridEvent> events = JsonConvert.DeserializeObject<List<EventGridEvent>>(jsonArray);
	foreach (var ev in events)
	{
	TriggeredFunctionData triggerData = new TriggeredFunctionData
	{
	TriggerValue = ev
	};
	await _listeners[functionName].Executor.TryExecuteAsync(triggerData, CancellationToken.None);
	}

In the current implementation, the EventGridExtensionConfig is always translating the incoming HttpRequestMessage body from JSON into a strongly typed List<EventGridEvent> and then triggering the functions with the individual EventGridEvent instances. Given that, the binding will never see string parameter.

So, that said, while the binding implementation supports converting the raw value from a string, it will never have to do that AFAICT and, therefore, the unit test that was added covers a scenario that doesn't really exist and should thus be removed. Additionally I would suggest that the test method is extremely overloaded with way too many responsibilities and should be refactored into multiple tests.

Support batched Event Grid Events

The programming model needs to have support for getting a collection of events as part of a single invocation.

public static void EventGridTest([EventGridTrigger] EventGridEvent[] eventGridEvents, ILogger log)
{
    ...
}

Keep in mind that this needs to work for other languages as well. Semantics for checkpointing / responding to Grid about completion need to be all-or-none.

See the EventHub trigger as an example.

Scenario-wise, part of this is to handle high-throughput. I can see a scenario where this is combined with Durable Functions to process through the batch, having checkpointed the whole batch with grid but capturing it in Durable

Documentation?

Could someone point me to any documentation about how to use the Event Grid extensions? I haven't found anything that gives any clues as to how it should be used!

I simply want to:

1. Create a subscription in my resource group's event topic (pre-existing) with the prefix I current need to use. This is done at startup my code's startup. If the subscription is already there, I presume I'll just re-use it. The code will supply or update the call-back URL that the Grid needs to use.

2. Respond to events from the Event Grid.

3. Send events to the Grid.

4. Stop listening (at shutdown).

Add authorization level to EventGridTrigger

Please provide a succinct description of the issue.

Repro steps

Provide the steps required to reproduce the problem

1. Create an EventGrid Trigger Azure Function.
2. You will notice that if you try to pass authLevel to the EventGridTrigger, it will not allow you because this class does not have any kind of overload.

Expected behavior

It should be configurable to use the Event Grid Trigger with auth schemes which are supported by the HttpTrigger.

Actual behavior

It doesn't exist.

Known workarounds

Create an HttpTrigger function and then redirect the Events to that endpoint.

EDIT -- Also when deployed, the only way to do something with the Event Grid Azure Function is to use the _master key. Other host keys don't work at all.

Azure Functions broken for NetCore3.1

I've upgraded to my Azure Function to netcoreapp3.1 targeting function v3. Since this change my EventGridTrigger attribute wont bind. It fails with:

System.Private.CoreLib: Exception while executing function: Function1. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'eventGridEvent'. Microsoft.Azure.WebJobs.Host: The method or operation is not implemented

Here are my dependencies:

<PackageReference Include="Microsoft.Azure.EventGrid" Version="3.2.0" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.EventGrid" Version="2.1.0" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.2" />

If you start a new function from Scratch in VS2019 you can reproduce in just a few mintues on localhost with postman.

Side note: if I use webhooks with 3.1 everything works fine. The only difference is the EventGrid extension.

Cannot get to work with pre-compiled C#

I have a project in VS 2017 targeting the v1 runtime. I have installed the nuget package Microsoft.Azure.WebJobs.Extensions.EventGrid 1.0.0-beta2.

The code (pasting below) builds fine, and publishes fine. I don't see any errors in logs. However it never fires. I create Event Grid functions in the portal in C# Script and they fire - though the data is the same. The closest thing I can see to a difference is when I use the "Test" button in the UX with the C# Script and pass in a string, the C# Script one that works throws an exception that it can't parse the event. The C# one (that doesn't work) says function fired and completed, but no exception is thrown.

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.EventGrid;
using Microsoft.Azure.WebJobs.Host;

namespace EventGridBlob
{
    public static class EventGridTrigger
    {
        [FunctionName("EventGridTrigger")]
        public static async Task RunAsync(
            [EventGridTrigger]EventGridEvent myEvent, 
            TraceWriter log)
        {
            log.Info($"Event Grid event recieved {myEvent.EventType}");
        }
    }
}

EventGrid Output binding error after converting to .net core 3.1

I have a function that uses the EventGrid output binding like so that has been working fine on our v2 function:

public async Task RunAsync( [EventGridTrigger] EventGridEvent eventGridEvent, [EventGrid(TopicEndpointUri = "EventGrid:Uri", TopicKeySetting = "EventGrid:SasKey")] IAsyncCollector<EventGridEvent> collector

After converting the function to .net core 3.1, I get the following error:

Unable to resolve app setting for property 'EventGridAttribute.TopicKeySetting'. Make sure the app setting exists and has a valid value

We made no other changes other than changing the runtime version to .net core 3.1.

Was there a change in the way the host works in 3.1 that wasn't tested with this output binding? I noticed that this output binding also isn't checked as supported(https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings#supported-bindings). What's the reason for that?

Event Grid Subscription for Event Hub to ingest logs in particular partition

While creating event subscription and using eventhub as endpoint there is no way of specifying the partitionID to which all messages should be sent. Is it possible via Webhook? I found that it is possible via Rest API see here. It uses the following url. https://{serviceNamespace}.servicebus.windows.net/{eventHubPath}/publishers/{deviceId}/messages. I am trying to ingest those events to event hub. but what I want is those events to be ingested into a particular partition in event hub.
Is it possible via Event subscription (probably by passing deviceid as parameter)? Or Is there any way to configure event subscription to route those events to a particular partitionID in event hub.

Subscription binding works via ngrok to a local machine, not with a function in Azure

Hi,
I have an Azure Function App that I can create a subscription to when it's hosted on a development machine via ngrok, but exactly the same function, when published to Azure, will not accept a subscription.

The error is "The attempt to validate the provided endpoint <> failed".
I have considered obvious things like the length of urls, but this does not seem to be the cause. The only difference between the two urls is the host, as in the host url of the function app, and the ngrok generated host. I have tried both using the portal and CLI to create subscriptions with the same result.

how do i get the systemkey for event grid trigger function endpoint

to create an azure function event grid subscription; i need the subscriber endpoint url.

i have tried to get the end point url following the steps mentioned here to no avail
https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-grid#create-a-subscription

someone kindly assist

https://{functionappname}.azurewebsites.net/admin/extensions/EventGridExtensionConfig?functionName={functionname}&code={systemkey}

what i need is the system key using powershell or ARM or Azure CLi

Dependency on Microsoft.Azure.WebJobs .NET 4.6

I'm trying to set up an Azure Function that has an EventGrid trigger in .NET 4.6

The Microsoft.NET.Sdk.Functions 1.0.13 has dependencies to Microsoft.Azure.WebJobs >= 2.2.0 && <2.3.0
The 2.0.0-beta1 extension requires Microsoft.Azure.WebJobs >= 3.0.0-beta5

Am I missing something, or can I not upgrade to the 2.0.0-beta1 version?

Connections keep open even if no request

What are these connections? Is it by design?
metrics screenshot can be found here:
https://share.weiyun.com/5fa2O4k

Redundant converter registration?

@watashiSHUN

I noticed this while looking at the code to refresh my memory on how to do something today:

// Register our extension binding providers
// use converterManager as a hashTable
// also take benefit of identity converter
context
    .AddBindingRule<EventGridTriggerAttribute>() // following converters are for EventGridTriggerAttribute only
    .AddConverter<JObject, string>((jobject) => jobject.ToString(Formatting.Indented))
    .AddConverter<string, JObject>((str) => JObject.Parse(str)) // used for direct invocation
    .AddConverter<JObject, EventGridEvent>((jobject) => jobject.ToObject<EventGridEvent>()) // surface the type to function runtime
    .AddOpenConverter<JObject, OpenType.Poco>(typeof(JObjectToPocoConverter<>))
    .BindToTrigger<JObject>(new EventGridTriggerAttributeBindingProvider(this));

// Register the output binding
var rule = context
    .AddBindingRule<EventGridAttribute>()
    .AddConverter<string, EventGridEvent>((str) => JsonConvert.DeserializeObject<EventGridEvent>(str))
    .AddConverter<JObject, EventGridEvent>((jobject) => jobject.ToObject<EventGridEvent>());
rule.BindToCollector(_converter);

Note how we have two entries for .AddConverter<JObject, EventGridEvent> - is there a difference in how these are utilized given one comes after registering a binding rule for a Trigger attribute, and the other comes after the Output attribute?

If not, can one of them be safely removed?

batching support

Azure/azure-webjobs-sdk@0bf125c

FunctionApp Systemkey: eventgrid_extension not created (quickly enough)

Hello,

With the recent ARM API update for Function Apps, I can query the system key necessary for making EventGrid subscriptions programmatically.
I've created a deployment procedure that automatically makes and deploys Function Apps and subscribes the functions to EventGrid topics.
The procedure does this:

• Create Function App resource
• Make local function app with azure-functions-core-tools
• Publish local function app with EventGridTrigger functions to Azure resource
• Obtain system key and subscribe Functions

The problem is that in the final step, the system key eventgrid_extension is not created yet - even though I did publish the function app with EventGrid triggered functions.
If I run the procedure again, the systemkey will exist the second time around and the subscriptions can be made.

Has anyone experienced this or knows why it happens? Another issue (#70) had the issue because there were no EventGridTrigger functions in the Function App - mine does have several, yet there is no systemkey yet.

Event Grid not delivery storage blob create events to function app running in Azure

Hello,

I have a storage account , Standard V2 LRS, , I created a subscription from the "Events" and selected endpoint type as Azure Function and selected an eventgrid triggered azure function. However when I try to upload new blob items, the event grid topic is reporting delivery failures -
outcome=Unauthorized, latencyInMs=1727, state=FilteredFailingDelivery

My function declaration is
[FunctionName(nameof(ProcessOrderBlobEventClient))]
public async Task Run(
[EventGridTrigger] EventGridEvent eventGridEvent,

net6.0 v4

Regards,
Sai

Error getting the SystemKey (404 or 401)

I am using the latest 2.0.0 version in the latest azure V2 runtime: 2.0.12115.0

I try to create an Event Grid SystemKey (to link my function to an Event Grid subscription) and I use the following code:
Invoke-RestMethod ("https://$functionAppName.azurewebsites.net/admin/host/systemkeys/eventgridextensionconfig_extension?code=" + $masterKey.masterKey.ToString())

I made a gist for the whole powershell script:
https://gist.github.com/olandese/b534226d2a08cf1e64d909c72be399a5#file-get-event-grid-systemkey-ps1

Steps to reproduce:

Create a V2 function with an EventGridTrigger.
Replace the $resourceGroupName and $functionAppName below the script and execute this after login in AzureRM in powershell.

When creating a function from the portal and executing the script i get a 404 when trying to retrieve the systemkey.
If I create a precompiled C# function (with an eventgridtrigger) and try to get te systemkey I get a 401.
This was all working fine until 2 days ago.

Event grid subscription not working for Storage Analytics containers?

I have created a event grid subscription with topic type as storage account. I want to trigger a Azure function as soon as any new logs get created. I also added a filter prefix /blobServices/default/containers/$logs/. Also logs in $logs container are created by Azure storage Analytics. Though the subscription creation is successful but azure function is not getting triggered.

Do we need to escape $logs? Or does event grid subscription does not work for special containers like $logs(which are hidden in ui)?

BlobCreatedEvent on Blob Copy

Hey folks,

I'm trying to understand how the event grid blob integration will behave when it comes to async blob copies. Here's what we want

1. Azure Function A starts an async blob copy from storage account A to B
2. Blob is copied to the new storage account A to B
3. Event Grid notifies Azure Function B that the copy is complete.

Will this scenario work given that the async blob copy creates an empty blob when the copy is initially triggered? We only want the event grid notification after the copy is complete. Is this feasible?

Thanks.

Subject filter doesn't work with cloud event schema subscription

=> This issue was created on request of @AzureSupport

It seems the subject filter doesn't work for Event Grid Subscriptions based on a Cloud Event Schema.
Interestingly enough, the exact set of steps does work correctly for 'Event Grid Schema' based ones.

Steps to reproduce:

1. Create new event grid topic instance, select Cloud Event schema
2. Create Event subscription, select Cloud Event schema and specify Storage Queue as Endpoint
3. Enable Subject filtering, set 'Subject begins with' to 'myapp' without the single quotes
4. Post event to Event Grid endpoint with this message:
   {
   "cloudEventsVersion": "0.1",
   "eventTypeVersion": "",
   "eventId": "12345",
   "source": "#/myapp/vehicles/motorcycles",
   "eventType": "queuea",
   "subject": "myapp/vehicles/hoedan",
   "eventTime": "2018-09-08T13:05:47.6120424Z",
   "data": {
   "make": "Honda",
   "model": "Monster"
   }
   }
5. Check Storage Queue, no event delivered.

Event Grid Trigger with Twilio

I'm getting this error when adding an Event Grid Trigger Azure function with Twilio:

The resource operation completed with terminal provisioning state 'Failed'. (Code: ResourceDeploymentFailure)
-The attempt to validate the provided endpoint htts://.......EventGridExtension.Config failed.

I have no issue with SendGrid. I wonder if Event Grid has any compatibility issue with Twilio.
Can anyone please send an example if you get it working?
Thanks!

EventGrid-triggered functions on the dev machines returns exception when UseDevelopmentStorage=false

When setting UseDevelopmentStorage=false in the local settings file and having a newly created event-grid triggered function the startup process fails with exception on this file/line below

BlobStorageSecretsRepository.cs
CloudStorageAccount account = CloudStorageAccount.Parse(_accountConnectionString);

Provide support for CloudEvents 1.0

Provide support for CloudEvents so that we are not forced to use Event Grid event schema.

Accept valid json as data payload for eventGridEvent

azure-functions-eventgrid-extension/src/EventGridExtension/EventGridTriggerAttributeBindingProvider.cs

Lines 111 to 114 in f13daea

	var bindingData = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase)
	{
	{ "data", (JObject)triggerValue["data"] }
	};

but we should deal with any JToken except JProperty

1. 'data':{...} (predefined, ie: blob event)
2. 'data':'...' (user defined event)
3. 'data':[..., ...]

Event Grid Messages "ABORTED".. Problem with Function or Event Grid?

I have an Azure function (dedicated App service plan) triggered by an Event Grid message when a file is created in Azure Blob Storage.

This seems to work very well in the small scale. However, when I add, say 1000 files to Azure Storage, it appears as if my Function isn't being triggered.
Investigating in EventGrid shows that the majority of those errors have a "GenericError" and are "Aborted"..

Looking at metrics for the Azure function, it doesn't appear that its hitting any kind of resource constraint (that I can tell). Memory/CPU/ all seem to be well below the capacity of my app service plan.

As i can't seem to drill any deeper into what an "Aborted" message is, and searching the web hasn't turned up much. I'm looking for help figuring out how to diagnose if this problem is in the configuration of my Function or in how I'm trying to use Event grid?

Thanks.

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

How to deploy a multi-region / DR configuration

Given that the extension uses a system key, and system keys can't be explicitly set, what solution(s) should be used to make a multi-region / disaster recovery configuration where the function app is deployed in multiple regions and a traffic manager or frontdoor solution is sending traffic to the active endpoint?

Exception handling and retries

I have been testing the EventGridTrigger and have a question/clarification.

When a function fails it is logged, but not retried from Event Grid. I guess this is by design?

If one would like to use the retry mechanisms in Event Grid, the regular HttpTrigger should be used instead?

Cheers

Weikko

EventGrid trigger throwing MethodNotImplemented exception.

Details about the issue here.
Azure/azure-functions-host#6328

The issue is not language specific. Confirmed that removing bundles, adding an eventgrid extension (2.1.0) was still causing the issue. Downgrading to 2.0.0 reverted to proper behavior.

Stack trace

System.NotImplementedException : The method or operation is not implemented. at async Microsoft.Azure.WebJobs.Host.Bindings.TriggerAdapterBindingProvider`2.ExactBinding`1.BindAsync[TAttribute,TTriggerValue,TUserType](??) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Bindings\BindingProviders\TriggerAdapterBindingProvider.cs : 213 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.TriggerWrapper.BindAsync(Object value,ValueBindingContext context) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Indexers\FunctionIndexer.cs : 479 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Host.Triggers.TriggeredFunctionBinding`1.BindCoreAsync[TTriggerValue](ValueBindingContext context,Object value,IDictionary`2 parameters) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Triggers\TriggeredFunctionBinding.cs : 57

Event Grid “Microsoft.EventHub.CaptureFileCreated” event not published to Azure Function subscription

I don't know if this is the right place for this issue, since it seems to affect other parts of Azure architecture, not only Azure Function bindings.

I am trying to develop an Azure Function to act upon a blob created by the Capture feature of an Event Hub. However, whereas capture blobs are correctly stored on the container, it seems that no Microsoft.EventHub.CaptureFileCreated event is published to the Function subscription. The event subscription to the function endpoint has been created without errors, and the output from the Azure CLI is

{
  "additionalProperties": {},
  "deadLetterDestination": null,
  "destination": {
    "additionalProperties": {},
    "endpointBaseUrl": "https://xxxxx-xxxxx-xxxxx.azurewebsites.net/api/EventGridWebhook",
    "endpointType": "WebHook",
    "endpointUrl": null
  },
  "eventDeliverySchema": "InputEventSchema",
  "filter": {
    "additionalProperties": {},
    "includedEventTypes": [
      "Microsoft.EventHub.CaptureFileCreated"
    ],
    "isSubjectCaseSensitive": null,
    "subjectBeginsWith": "",
    "subjectEndsWith": ""
  },
  "id": "/subscriptions/yyyyy-yyyyy-yyyyyy/resourceGroups/test-event-grid-grp/providers/Microsoft.EventHub/namespaces/capture-hub-namespace/providers/Microsoft.EventGrid/eventSubscriptions/captureFunctionV1Sub",
  "labels": null,
  "name": "captureFunctionV1Sub",
  "provisioningState": "Succeeded",
  "resourceGroup": "test-event-grid-grp",
  "retryPolicy": {
    "additionalProperties": {},
    "eventTimeToLiveInMinutes": 1440,
    "maxDeliveryAttempts": 30
  },
  "topic": "/subscriptions/yyyyy-yyyyy-yyyyyy/resourceGroups/test-event-grid-grp/providers/microsoft.eventhub/namespaces/capture-hub-namespace",
  "type": "Microsoft.EventGrid/eventSubscriptions"
}

The body of the function is a standard Http trigger with the additional validation part needed for the Event Grid endpoint subscription

public static async Task<object> Run(HttpRequestMessage req, TraceWriter log)
{
    string jsonContent = await req.Content.ReadAsStringAsync();
    log.Info($"Webhook was triggered! Input: {jsonContent}");
    dynamic events = JsonConvert.DeserializeObject(jsonContent);

    if (req.Headers.GetValues("Aeg-Event-Type").FirstOrDefault() == "SubscriptionValidation")
    {
        var code = events[0].Data["validationCode"];
        return req.CreateResponse(HttpStatusCode.OK,
            new { validationResponse = code });
    }

    return req.CreateResponse(HttpStatusCode.OK);
}

If I add the same function's endpoint for another type of event (f.i. a blob creation event), I can see the function invocations in the logs. Besides, in the Metrics blade of the event subscription, it seems like no event is ever published to the subscribers (no published, no failed, no unmatched).

Also, it doesn't work with either of the two versions of the runtime environment and even the EventGridTrigger attribute doesn't work.

The same question (with more details) is available on stackoverflow.