Data Engineering on Microsoft Azure Test 13

Which of the following may be used to accomplish this?

Select 1 option(s):

Select 1 option(s):

Data processing within Hadoop uses which of the following to process big data? (Select three)

Select 3 option(s):

• Too many small or very big files – more time opening & closing files rather than reading contents (worse with streaming).
• Partitioning also known as “poor man’s indexing”- breaks down if you picked the wrong fields or when data has many dimensions, high cardinality columns.
• No caching – cloud storage throughput is low (cloud object storage is 20-50MB/s/core vs 300MB/s/core for local SSDs).
As a solution to the challenges with Data Lakes noted above, Delta Lake is a file format that can help you build a data lake comprised of one or many tables in Delta Lake format. Delta Lake integrates tightly with Apache Spark, and uses an open format that is based on Parquet.
Two of the core features of Delta Lake are performing UPSERTs and Time Travel operations.
What does the UPSERT command do?

Select 1 option(s):

Which of the Azure Storage account options is best described by the following:
“A legacy account type which supports only block and append blobs.”

Select 1 option(s):

One of the current projects is geared towards the design of an enterprise data warehouse in Azure Synapse Analytics.
Specifications:
• There will be millions of rows of data loaded to the data warehouse daily
• Staging tables must be optimized for data loading
Required:
• Design the staging tables
The team has come up with a list of options they are considering to use for the task at had but are not sure which to utilize. As you are the Azure expert, the team leader has asked for your opinion on which to use.
Which of the following should you recommend for the staging table design?

Select 1 option(s):

Select 1 option(s):

[A] data is typically tabular data that is represented by rows and columns in a database. Databases that hold tables in this form are called [B] databases.

Select 1 option(s):

Conditional Access policies at their simplest are [?].

Select 1 option(s):

Requirement: A globally distributed, multimodel database that can perform queries in less than a second.
Which of the following should you choose?

Select 1 option(s):

A common use with [?] is to take the data that is shared and use it as a source into Azure Data Factory pipelines to use with your own internal data.

Select 1 option(s):

Azure Synapse Analytics can work by acting as the one stop shop to meet all of your analytical needs in an integrated environment.
[?] leverages the capabilities of Azure Data Factory and is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using this, you can create and schedule data-driven workflows that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, or Azure Databricks.

Select 1 option(s):

Select 1 option(s):

What does DevOps focus on?

Select 1 option(s):

Select 1 option(s):

Many business application architectures separate transactional and analytical processing into separate systems with data stored and processed on separate infrastructures. [?] systems are optimized for the analytical processing, ingesting, synthesizing, and managing large sets of historical data.

Select 1 option(s):

While ACME is happy with the capabilities and performance of Azure Cosmos DB, they are concerned about the cost of executing a large volume of analytical queries over multiple partitions (cross-partition queries) from their data warehouse. They want to efficiently access all the data without needing to increase the Azure Cosmos DB request units (RUs).
They have looked at options for extracting data from their containers to the data lake as it changes, through the Azure Cosmos DB change feed mechanism. The problem with this approach is the extra service and code dependencies and long-term maintenance of the solution. They could perform bulk exports from a Synapse Pipeline, but then they won’t have the most up-to-date information at any given moment.
Required: Pick a solution will ensure all transactional data is automatically stored in a fully isolated column store without impacting the transactional workloads or incurring resource unit (RU) costs.

Select 1 option(s):

In Data Lake Storage Gen1, data engineers query data by using the [?] language.

Select 1 option(s):

Select 1 option(s):

Specifications:
• The dimension table will be less than 1 GB.
Required:
•  Fastest available query time
•  Minimize data movement
As the Azure expert, the O’Shaughnessy IT team looks to you for direction. Which of the following should you advise them to utilize?

Select 1 option(s):

Which are approaches to processing data streams? (Select two)

Select 2 option(s):

The company has 10,000 employees. Most employees are located in Europe. The company supports teams worldwide.
AIM has two main locations: a main office in London, England, and a manufacturing plant in Berlin, Germany.
During events, 100 engineers set up a remote portable office by using a VPN to connect the datacentre in the London office. The portable office is set up and torn down in approximately 20 different countries each year.
Chaos Central
During major events, AIM uses a primary application named Chaos Central. Each vehicle used in the activity has several sensors that send real-time telemetry data to the London datacentre. The data is used for real-time tracking of the vehicles. Chaos Central also sends batch updates to an application named Mechanical Workflow by using Microsoft SQL Server Integration Services (SSIS).
The telemetry data is sent to a MongoDB database. A custom application then moves the data to databases in SQL Server 2017. The telemetry data in MongoDB has more than 500 attributes. The application changes the attribute names when the data is moved to SQL Server 2017.
The database structure contains both OLAP and OLTP databases.
Mechanical Workflow
Mechanical Workflow is used to track changes and improvements made to the vehicles during their lifetime. Currently, Mechanical Workflow runs on SQL Server 2017 as an OLAP system. Mechanical Workflow has a named Table1 that is 1 TB. Large aggregations are performed on a single column of Table 1.
Requirements:
• Data collection for Chaos Central must be moved to Azure Cosmos DB and Azure SQL Database. The data must be written to the Azure datacentre closest to each race and must converge in the least amount of time.
• The query performance of Chaos Central must be stable, and the administrative time it takes to perform optimizations must be minimized.
• The datacentre for Mechanical Workflow must be moved to Azure SQL data Warehouse.
• Transparent data encryption (TDE) must be enabled on all data stores, whenever possible.
• An Azure Data Factory pipeline must be used to move data from Cosmos DB to SQL Database for Chaos Central. If the data load takes longer than 20 minutes, configuration changes must be made to Data Factory.
• The telemetry data must migrate toward a solution that is native to Azure.
• The telemetry data must be monitored for performance issues. You must adjust the Cosmos DB Request Units per second (RU/s) to maintain a performance SLA while minimizing the cost of the Ru/s.
Which of the following data stores should be configured to meet the TDE requirement?

Select 1 option(s):

Spark has optimized this operation by using a format called [?] which prevents the need for expensive serialization and de-serialization of objects in order to get data from one JVM to another.

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Which of the following attributes belong to Spark Instances? (Select three)

Select 3 option(s):

Select 4 option(s):

A storage account defines a policy that applies to all the storage services in the account.
Which are the settings that are controlled by a storage account. (Select all that apply)

Select 6 option(s):

Select 1 option(s):

[?] is a fully-managed, cloud-based Big Data and Machine Learning platform, which empowers developers to accelerate AI and innovation by simplifying the process of building enterprise-grade production data applications.

Select 1 option(s):

When working with large data sets, it can take a long time to run the sort of queries that clients need. These queries can’t be performed in real time, and often require algorithms such as MapReduce that operate in parallel across the entire data set. The results are then stored separately from the raw data and used for querying.
One drawback to this approach is that it introduces latency. If processing takes a few hours, a query may return results that are several hours old. Ideally, you would like to get some results in real time (perhaps with some loss of accuracy), and combine these results with the results from the batch analytics.
The [?] is a big data processing architecture that addresses this problem by combining both batch- and real-time processing methods. It features an append-only immutable data source that serves as system of record. Timestamped events are appended to existing events (nothing is overwritten). Data is implicitly ordered by time of arrival.

Select 1 option(s):

Select 4 option(s):

Select 1 option(s):

Required: The team is to implement masking for the Target_ID field as per the following:
• Set the initial three prefix characters as “exposed”.
• Set the final three suffix characters as “exposed”.
• The remaining characters as “masked”.
The team is planning to utilize data masking with a credit card function mask.
Will this solution meet the requirements?

Select 1 option(s):

Design A:

Design B:

Design C:

Which architecture would be best suited for the need?

Select 1 option(s):

Which of the following are ingestion methods that can be used to extract data from a variety of sources?

Select 3 option(s):

Select 1 option(s):

Which transformation in Mapping Data Flow is used to do this?

Select 1 option(s):

Azure Synapse Analytics is an integrated analytics platform, which combines data warehousing, big data analytics, data integration, and visualization into a single environment. Azure Synapse Analytics empowers users of all abilities to gain access and quick insights across all of their data, enabling a whole new level of performance and scale.
Descriptive analytics answers the question [?].

Select 1 option(s):

Select 1 option(s):

Clicking Debug will provision the Spark clusters required to interact with the Mapping Data Flow transformations. If you select AutoResolveIntegrationRuntime, what will be the result? (Select all that apply)

Select 2 option(s):

Bruce has come to you because his IT team needs advice on which API to use for the database model and type based on the following information.
Specifications:
• The application uses a NoSQL database to store data
• The database uses the key-value and wide-column NoSQL database type.
Required: Developers need to access data in the database using an API.
Which of the following APIs should you recommend to Bruce and his team?

Select 2 option(s):

Synapse dedicated SQL Pools supports JSON format data to be stored using [?]. The JSON format enables representation of complex or hierarchical data structures in tables. It allows to transform arrays of JSON objects into table format. The performance of JSON data can be optimized by using columnstore indexes and memory optimized tables.

Select 1 option(s):

Azure Synapse Analytics is a cloud-based data platform that brings together enterprise data warehousing and Big Data analytics. It can process massive amounts of data and answer complex business questions with limitless scale.
Azure Synapse Analytics uses the [?] approach for bulk data.

Select 1 option(s):

Select 1 option(s):

What will the result be of pipeline?

Select 1 option(s):

As a Data Engineer, you can transfer and move data in several ways. The most common tool is [?], which provides robust resources and nearly 100 enterprise connectors. [?] also allows you to transform data by using a wide variety of languages.

Select 1 option(s):

Select 1 option(s):

• Data movement activities
• Data transformation activities
• Control activities
A Control Activity in Data Factory is defined in JSON format as follows:
JSON
{
“name”: “Control Activity Name”,
“description”: “description”,
“type”: “”,
“typeProperties”:
{
},
“dependsOn”:
{
}
}
Which of the JSON properties are required? (Select all that apply)

Select 3 option(s):

You can programmatically define a linked service in the JSON format to be used via REST APIs or the SDK, using the following notation:
JSON
{
“name”: ““,
“properties”: {
“type”: ““,
“typeProperties”: {
“”
},
“connectVia”: {
“referenceName”: ““,
“type”: “IntegrationRuntimeReference”
}
}
}
Which of the JSON properties are required? (Select all that apply)

Select 3 option(s):

Below is a sample overview of the CI/CD lifecycle in an Azure data factory that’s configured with Azure Repos Git.
The order of the activities has been shuffled.
a. A developer creates a feature branch to make a change. They debug their pipeline runs with their most recent changes.
b. A development data factory is created and configured with Azure Repos Git. All developers should have permission to author Data Factory resources like pipelines and datasets.
c. After a pull request is approved and changes are merged in the master branch, the changes get published to the development factory.
d. After the changes have been verified in the test factory, deploy to the production factory by using the next task of the pipelines release.
e. When the team is ready to deploy the changes to a test or UAT (User Acceptance Testing) factory, the team goes to their Azure Pipelines release and deploys the desired version of the development factory to UAT. This deployment takes place as part of an Azure Pipelines task and uses Resource Manager template parameters to apply the appropriate configuration.
f. After a developer is satisfied with their changes, they create a pull request from their feature branch to the master or collaboration branch to get their changes reviewed by peers.
Select the correct sequence of events in the CI/CD lifecycle.

Select 1 option(s):

Which of the following are examples of these opportunities? (Select all that apply)

Select 4 option(s):

Dedicated SQL Pools have which of the following indexing options available?

Select 3 option(s):

Which of the following are the referenced hubs on Azure Synapse Studio? (Select six)

Select 6 option(s):

Select 1 option(s):

You can use the [?] to perform an assessment of the SSIS packages that exist and identify any compatibility issues with them.

Select 1 option(s):

Select 1 option(s):

You can use account-level SAS to allow access to anything that a service-level SAS can allow, plus additional resources and abilities. You’d use this type of SAS, for example, … [?] (Select all that apply)

Select 1 option(s):

[?] is typically used to automate the process of extracting, transforming, and loading the data through a batch process against structured and unstructured data sources.

Select 1 option(s):

Select 1 option(s):

Which of the following is best described by:
“By placing the init script in /databricks/init folder, you force the script’s execution every time any cluster is created or restarted by users of the workspace.”

Select 1 option(s):

To provide a better authoring experience, Azure Data Factory allows you to configure a Git repository with either Azure Repos or GitHub. Git is a version control system that allows for easier change tracking and collaboration.
True or False: Configuring a git repository allows you to save changes, letting you only publish when you have tested your changes to your satisfaction.

Select 1 option(s):

Select 1 option(s):

Select 3 option(s):

Select 1 option(s):