【Azure Developer】使用REST API获取Activity Logs、传入Data Lake的数据格式问题

问题一:.  如何在用REST API获取活动日志时,控制输出的项?

【答】参考REST API对于获取活动日志的说明接口,在参数是**$filter$select中可以分别控制过滤条件输出项**

GET https://management.chinacloudapi.cn/subscriptions/{subscriptionId}/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-01&$filter={$filter}&$select={$select}

注:management.chinacloudapi.cn为中国区Azure的管理终结点,management.azure.com为全球Azure的管理终结点

URI Parameters 

Name Description
$filter Reduces the set of data collected. This argument is required and it also requires at least the start date/time. The $filter argument is very restricted and allows only the following patterns. - List events for a resource group: $filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceGroupName eq 'resourceGroupName'. - List events for resource: $filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceUri eq 'resourceURI'. - List events for a subscription in a time range: $filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z'. - List events for a resource provider: $filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceProvider eq 'resourceProviderName'. - List events for a correlation Id: $filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and correlationId eq 'correlationID'. NOTE: No other syntax is allowed.
$select Used to fetch events with only the given properties. The $select argument is a comma separated list of property names to be returned. Possible values are: authorization, claims, correlationId, description, eventDataId, eventName, eventTimestamp, httpRequest, level, operationId, operationName, properties, resourceGroupName, resourceProviderName, resourceId, status, submissionTimestamp, subStatus, subscriptionId

在Filter和select可以配置的每一项说明,请查看文末的附录一

 

问题二:是否可以往Data Lake中注入Json格式文件?

【答】根据官方文档示例,如果使用Databricks进行(SQL查询语句)进行数据分析,是支持Json格式的数据,示例中使用的Json格式如下图所示:

参考使用 Databricks 分析数据一文: https://docs.azure.cn/zh-cn/storage/blobs/data-lake-storage-quickstart-create-databricks-account#ingest-sample-data

 

问题三:如何用C#向Data Lake中插入数据文件?

【答】可以,需要先安装 Azure.Storage.Files.DataLake NuGet 包。然后添加如下引用

using Azure.Storage.Files.DataLake;
using Azure.Storage.Files.DataLake.Models;
using Azure.Storage;
using System.IO;
using Azure;

将文件上传到目录,则参考以下代码即可:

public async Task UploadFile(DataLakeFileSystemClient fileSystemClient)
{
    DataLakeDirectoryClient directoryClient =
        fileSystemClient.GetDirectoryClient("my-directory");

    DataLakeFileClient fileClient = await directoryClient.CreateFileAsync("uploaded-file.txt");

    FileStream fileStream = 
        File.OpenRead("C:\\file-to-upload.txt");

    long fileSize = fileStream.Length;

    await fileClient.AppendAsync(fileStream, offset: 0);

    await fileClient.FlushAsync(position: fileSize);

}

完整实例参考使用 .NET 管理 Azure Data Lake Storage Gen2https://docs.azure.cn/zh-cn/storage/blobs/data-lake-storage-directory-file-acl-dotnet#upload-a-file-to-a-directory

 

 

附录一:在$Filter和$select可配置的选择项的EventData属性

The Azure event log entries are of type EventData

Name Type Description
authorization SenderAuthorization The sender authorization information.
caller string the email address of the user who has performed the operation, the UPN claim or SPN claim based on availability.
category LocalizableString the event category.
claims object key value pairs to identify ARM permissions.
correlationId string the correlation Id, usually a GUID in the string format. The correlation Id is shared among the events that belong to the same uber operation.
description string the description of the event.
eventDataId string the event data Id. This is a unique identifier for an event.
eventName LocalizableString the event name. This value should not be confused with OperationName. For practical purposes, OperationName might be more appealing to end users.
eventTimestamp string the timestamp of when the event was generated by the Azure service processing the request corresponding the event. It in ISO 8601 format.
httpRequest HttpRequestInfo the HTTP request info. Usually includes the 'clientRequestId', 'clientIpAddress' (IP address of the user who initiated the event) and 'method' (HTTP method e.g. PUT).
id string the Id of this event as required by ARM for RBAC. It contains the EventDataID and a timestamp information.
level EventLevel the event level
operationId string It is usually a GUID shared among the events corresponding to single operation. This value should not be confused with EventName.
operationName LocalizableString the operation name.
properties object the set of <Key, Value> pairs (usually a Dictionary<String, String>) that includes details about the event.
resourceGroupName string the resource group name of the impacted resource.
resourceId string the resource uri that uniquely identifies the resource that caused this event.
resourceProviderName LocalizableString the resource provider name of the impacted resource.
resourceType LocalizableString the resource type
status LocalizableString a string describing the status of the operation. Some typical values are: Started, In progress, Succeeded, Failed, Resolved.
subStatus LocalizableString the event sub status. Most of the time, when included, this captures the HTTP status code of the REST call. Common values are: OK (HTTP Status Code: 200), Created (HTTP Status Code: 201), Accepted (HTTP Status Code: 202), No Content (HTTP Status Code: 204), Bad Request(HTTP Status Code: 400), Not Found (HTTP Status Code: 404), Conflict (HTTP Status Code: 409), Internal Server Error (HTTP Status Code: 500), Service Unavailable (HTTP Status Code:503), Gateway Timeout (HTTP Status Code: 504)
submissionTimestamp string the timestamp of when the event became available for querying via this API. It is in ISO 8601 format. This value should not be confused eventTimestamp. As there might be a delay between the occurrence time of the event, and the time that the event is submitted to the Azure logging infrastructure.
subscriptionId string the Azure subscription Id usually a GUID.
tenantId string the Azure tenant Id

 

 

正在加载评论...