All posts by cseferlis

AAD, Azure, Infrastructure, Security

What is Azure Active Directory B2C?

Leave a comment

How important is secure identity management to you? If you’re like most, it is a top priority. In today’s post I’ll talk about Azure Active Directory B2C which is an identity management service that enables you to customize and control how users securely interact with your web, desktop, mobile or even single applications.

Using Azure AD B2C, users can sign up, sign in, reset passwords and edit profiles for the various applications they’re using.

When implementing these policies, we’ll have two choices:

  • Using common identity user flows within the Azure portal or,
  • For the more skilled developer or if the templates in the portal don’t support your use case, you can use XML based custom policies.

Once you make that decision, your choice will define the path of authentication, commonly referred to as the user journey. User journeys allow you to control behaviors by configuring some settings; things like social accounts (like Facebook) that the user uses to sign up for the application.

Data collected from the user as a first name or postal code would be used for authentication. You also have multi-factor authentication options, as well as the look and feel of how users interact with pages and information returned to the application.

Azure Active Directory B2C supports the open ID connect and the OAuth 2 protocols for these user journeys. These protocols will help ultimately receive a token that will allow for you to be authenticated. The interaction of every application follows a similar high-level pattern shown in the graphic below:

AAD B2C Flow

The steps here are:

1. The application directs the user to run a policy.

2. The user completes the policy according to the policy definition.

3. Then the application receives a token.

4. And then uses that token to try to a resource.

5. The resource server then validates the token to verify that access can be granted.

6. And the application will periodically refresh in the background ( there really are 5 steps but this 6th step is happening over and over).

Azure AD B2C can also work with additional identity providers such as Amazon, Facebook and Google that will create, maintain and manage identity information while providing authentication services to their (and other) applications.

Typically, you would only use one identity provider in your application but there are no restrictions for using more if your use case calls for it.

The main value for this service is the ability to lessen the need for username and password management for so many applications, thus improving the user experience. Our lives have been made a bit easier since we now have many applications, both web and desktop based, that allow that single sign on or no sign on experience because they are already pre-authenticated with a service like this.

AAD, Azure, Infrastructure, Security, SSO

What is Azure Active Directory Seamless Single Sign On?

Leave a comment

We’re all dealing with many usernames and passwords in our everyday life, right? Today I’d like to talk about an authentication feature within Azure Active Directory that can help you with easier, faster access.

Azure Active Directory Seamless Single Sign-on (Azure AD Seamless SSO) automatically signs users in when they are on their corporate devices connected to their corporate network. When this is enabled, users don’t have to type their passwords, or even their username, to sign in to Azure Active Directory.

This feature provides users with easy access to cloud-based applications without needing any additional on premises components.

First let’s discuss how this is set up:

  • SSO is enabled used Azure AD Connect. The following steps will occur while enabling this feature:
    • A computer account representing Azure AD is created in your on premises Active Directory in each AD forest.
    • The computer account Kerberos decryption key is shared securely with Azure AD and then 2 Kerberos service principal names (SPNs) are created to represent 2 URLs that are used during Azure AD sign-on.

Authenticating in Browser

  • When doing authentication from a web browser for a web app, essentially a user navigates to a website and signs into Azure AD (see below).
AADSSSO-Image 1
  • Azure AD sends a Kerberos requests to on premises AD and on premises AD looks for an account related to the device you’re signing in on and a user account. If authorized, you get access.

Authenticating with Native Application

  • For a native client, like Outlook for instance, the process is a bit different (see below).
AADSSSO-Image 2
  • Here, the request is made from the device you’re using and authenticated off Azure AD, issuing a Kerberos ticket when it is successful.
  • When that ticket is authenticated off Azure AD and approved, a SAML token is sent to the app. Then it gets sent back to AAD for OAuth-2 authentication.
  • Once all that checks out, access is granted.

Now let’s talk about the benefits.

  • First, it’s a much better user experience. Users are automatically signed in both on premises and cloud-based applications using their built-in authentication, so there’s no need for users to repeatedly reenter their passwords.
  • It’s also easy to deploy and administer. There are no additional components needed on premises; it synchronizes your Azure Active Directory to your AD. Plus it works with any method of cloud authentication using password hash synchronization or pass through authentication.
  • Additionally, it can be rolled out to only some or all of your users by using group policy.

So, this is a great way to allow users to have multiple authentications into multiple websites and applications using only one authentication tool. This will minimize the amount of administration required to set up those users once it’s in place. And it should reduce the number of password resets for your help desk team or whomever oversees that.

Azure, Azure Data Factory, Database, Infrastructure

How and Why to Add a Source Code Repository to Azure Data Factory

Leave a comment

For developers, it’s very beneficial to have a source code repository. A source code repository helps to keep all your changes, to manage tasks, branches, share the code with a team and simply put, to keep it in safe place.

In this post, I’ll tell you why you should connect your Azure Data Factories to a source code repository, and I’ll demo how to do so:

  • To do this, I’ll start in my data factory inside my Azure portal.
  • When I go into Author & Monitor, I have the ability to either:
    • Set up a code repository within the landing page or main page here or,
    • I can go directly into my data factory and I can add it in the left-hand corner and pull down to where it says, ‘set up code repository’.
  • One thing to note is the code repository itself has been supported for a while, but recently with the release of Data Flows, they’re now supporting GitHub within the repositories as well (previously is was only Azure DevOps).
  • So, I select my GitHub account and fill in the information. *If you’re doing this for the first time, it’s going to prompt you to log into your GitHub account when you do this. In this case, I’ve already previously connected this so it’s going to know about my repositories.
  • Next, I select my repository name and I’ll go to my playground branch and I’m going to use my existing playground.
  • One field will ask me ‘Branch to import resources into:’ so if I’m importing resources, I can select an existing one or create a new one. For this demo I’m going to pick my playground.
  • Before I hit Save, notice on the left-hand side I’ve got zero pipelines, one data set and zero data flows. But when I connect to my playground, it’s going to bring everything in I’ve previously connected to within any of my areas I’ve saved up into or checked my code into anything in that playground branch. So, now you’ll see I have 6 pipelines, 23 data sets and 4 data flows.
  • One of the other nice pieces of being able to add source control is if I want to add a new data set. I just select my SQL Server I was previously connected to; I leave it on default for now and connect in.
  • I then select one of the tables; I selected the Product Category Table.
  • You’ll see at the top you have the option to Save All or Publish. Save All is going to save any of the changes across the tabs, so you can tell when there’s been a change, whether it be data set or pipeline or data flow by having a star next to the name.
  • Now instead of needing to publish every time you’re doing development, you can just save it and it will save here. So, rather than having to publish the entire pipeline and do any of the error checking and make sure the pipeline is in good standing, you can now just save part way through. This is a huge advantage over having to publish the entire pipeline which could cause some challenges which might not be efficient for development and such.

The new features of GitHub being added in gives us another great opportunity if you didn’t previously use Azure Dev Ops (formerly known as Visual Studio Team Services). I’ll be doing more upcoming blogs around Data Factory in Azure Every Day that will be beneficial to you with some of the nuances as the product has greatly evolved since releasing Data Flows.

Azure, Infrastructure, IoT, Networking

Expanding the Azure Data Box Family

Leave a comment

In a previous blog I introduced Azure Data Box. Today I’d like to talk about how Microsoft is expanding the Azure Data Box family by introducing you to the Azure Data Box Gateway and the Azure Data Box Edge devices.

Until now the Data Box Family has been the disc, the box and the heavy. Each have their own limits for storage but are designed to improve your way of uploading massive amounts of data into Azure, without having to wait for it to travel across the wire or saturate your bandwidth (consider that the offline method).

Microsoft learned from customers that they want a better way to sync their local storage directly with Azure storage for operations like archival and disaster recovery. Here’s where Azure Data Box Gateway comes in.

The Data Box Gateway is a cloud storage gateway device that resides on premises and sends your image, media and other data directly to Azure.

  • The Gateway is a virtual machine provisioned in your Hypervisor (VMware or Hyper V) where you write the data directly to this virtual device using the NFS or SMB protocols, which it then sends to Azure.
  • One use case for the Data Box Gateway is for things like continuously ingesting massive amounts of data. So, we have a local data source that requires large data amounts and capacities and we can stream those and sync them directly with our Azure storage.
  • Another use case would be for a cloud archival of data in a secure and efficient way. If you then think about the incremental data transfer over the network after the initial bulk transfer is done using the Data Box of your choice for direct tie in to the same Azure storage container that you’re using for your Data Box.

Azure Data Box Edge is a storage solution that allows you to process data and send it over the network to Azure.

  • Data Box Edge uses a physical device supplied by Microsoft to accelerate the secure data transfer.
  • The device resides on premises in your network stack and you write data to it (also using NFS or SMB.)
  • It is additionally equipped with AI enabled Edge computing capabilities which help to analyze, process or filter data as it moves to Azure block blob, page blob or Azure files.
  • It has the appropriate chips to process intelligent learning (artificial intelligence, machine learning, deep learning and such).
  • Use cases are for things like pre-processing data. So, we can analyze data from on premises or IOT devices to get faster information about the data. That pre-processing will allow us to do things like aggregating your data before it gets sent to Azure or modifying data, such as taking out PII.
  • You can also subset and transfer the data needed for deeper analytics in the cloud.
  • Additionally, you can analyze and react to IOT events. So, if you’re running IOT devices on prem and you want the ability to be quicker to respond when those events occur, this is a great way to handle that.
  • Another great use case is you can run machine learning models to get quick results that can be acted on before the data is sent to the cloud.
  • With these IOT use cases, you don’t have to wait for the data to be transmitted over the wire, do any of the munging happening up in Azure and then return results. You can return those results on the fly in real time and react more quickly.
  • Eventually the full data set is transferred to continue and help you to retain and improve any of your machine learning models. You can continually feed it data and have those models trained repeatedly, thus learning to be more concise over time.

The Data Box family is a very cool technology by having an online version to further extend its capabilities.

Azure, Compliance, Security, Strategy

5 Ways Azure Makes Your Enterprise More Secure

Leave a comment

Security is, or should be, a top priority; nothing is more important than making your enterprise secure. In this post I’ll tell you 5 ways Azure makes your enterprise more secure.

First off, Azure is a Microsoft product. When you’re one of the world’s largest companies, there are an enormous amount of threats that need to be evaluated every second of the day. So, obviously Microsoft is aware of these challenges.

With that in mind, Microsoft developed centers of excellence over the past ten years in order to be ready for these attacks. The Microsoft Threat Intelligence Center processed over 6.5 trillion signals so they could better understand what kind of information and what types of attack vendors there are.

Each month they block over 5 billion distinct malware threats. And they staff over 3500 security professionals in their defense operations centers to help thwart these attacks. Since Active Directory is a standard for user authentication control, they introduced Azure Active Directory years ago to extend that to their Azure platform.

All that being said, here are 5 ways that Azure makes your enterprise more secure:

1. Minimize the requirement for password use – By using Microsoft Authenticator and connecting to Software as a Service applications (like Drop Box, Salesforce, etc.) The authenticator replaces your password with a multi-factor sign in using something like your phone and your fingerprint, face ID or a pin based on the Windows device that you’re using.

With a 2-factor authentication when using those devices, you have a more simplified method instead of remembering a bunch of different passwords.

2. Security Scorecard – A while back I did a post on the Azure Secure Score and the Secure Score Center. With this, you’re using the Azure portal for having awareness where there are potentials for exposure or for best practices that need to be followed which helps your organization stay better secured.

3. Microsoft Threat Protection Suite – Helps detect, investigate and remediate issues across your organization, including endpoints, email, documents, identity and infrastructure elements. It also helps your security team automate many of those manual, mundane security tasks.

4. Confidentiality – Microsoft was the first cloud vendor to introduce confidential integrity in data while it’s in use. So, consumers don’t worry about their data being put in the wrong hands (like some of those other clouds vendors you may have heard of recently in the news).

Data is always encrypted at rest and in transit. The security will soon extend to the chip level for added security on certain Azure VMs. Intel has built in some security measures inside their chips and now Microsoft is going to interact directly with those chips to keep you more secure.

5. Microsoft Information Protection Service – This enables you to automatically discover, classify, label, protect and monitor data no matter where it lives or travels on your Microsoft devices.

We’re now seeing many more open source capabilities and seeing more of these applications being sent over to Macs and Linux PCs for instance. Essentially this labeling capability is built into office apps and such across all the major platforms and can add protection capability to things like PDF documents, a feature currently in preview.

But the idea is it’s going to help you protect from things such as PII being extended. So, it’s an added level of protection to ensure there are no security leaks.

So, it’s clear from all this that Microsoft not only has a commitment to securing their own services and software, but also enterprises and individuals are of critical importance when talking about security.

If you’re concerned about security, check out some of the things I mentioned here and remember, Microsoft is making the investment and doing all they can to keep things secure.

Azure, Azure Cognitive Services, Big Data

Microsoft and BlackRock Announce Retirement Planning Partnership

Leave a comment

At this point, the state of financial planning is a potential major crisis with current and future generations coming upon retirement age with little or no savings to account for.

As we’ve moved away from pensions of the old, the responsibility held previously by the companies, has now shifted to individuals having to invest and save on their own to ensure they’re set up after they retire.

I wanted to share a recent press release from Microsoft who announced that they created a partnership with BlackRock to help reimagine the way people manage their retirement planning. BlackRock is a world leader in wealth management, including providing solutions to consumers and currently manages approximately 6.5 trillion in assets for investors worldwide.

The goal of this alliance is to find ways for people to interact with their retirement assets more, so they know what kind of contributions they’re making. BlackRock will design and manage a suite of next generation investment tools that aim to provide a ‘lifetime’ of income in retirement. This would be made available to US workers through their employer’s workplace savings plan.

The press release did not share much detail about what exactly the two firms will partner on, but the following is a quote from Microsoft CEO, Satya Nadella: “Together with BlackRock, we will apply the power of the cloud and AI to introduce new solutions that address this important challenge and reimagine retirement planning.”

As we know, AI, deep learning and machine learning and all their related technologies, can have a profound impact on information gathering, processing and the intelligence we can extract from it. This helps us make better decisions.

The idea here is to offer technology options to businesses for their employees to consume and promote fiduciary responsibility. There will be more complex options that have been shunned previously by employers because of their complexity and costliness.

BlackRock has shown that they want to move their technology footprint forward with acquisitions and investments in firms in recent years. In 2015, they acquired a robo advisor company, as well as invested in Acorns, a company which helps millennials save their spare change to put it into a savings account.

Last year, BlackRock acquired Investment, a company that gives them more sophisticated online investment tooling. It is also believed that additional partnerships will come along to help support any of these new investment options, the plans and the employees.

When it comes to how the world is changing, AI is thought to be one of the biggest conversations occurring in 2019. At the heart of AI is data—data quality and consistency. These important factors are something we focus on at Pragmatic Works, as well as knowing that this is what our clients need to rely on.

This press release shows where we’re going with some of the AI technology that’s a huge topic of conversations in organizations today.

Azure, Big Data, Database, Databricks

3 Common Analytics Use Cases for Azure Databricks

Leave a comment

Pragmatic Works is considered to be experts in the Microsoft Data Platform, both on-premises and in Azure. That being said, we often get asked many questions like, how can a certain technology benefit my company? One technology we are asked about a lot is Azure Databricks. This was released over a year ago in preview in the Azure portal and we’re starting to see some massive adoption by many companies, but not everyone is ready to delve into data science and deep analytics, so they haven’t had much exposure to what Databricks is and what it can do for their business.

There are some barriers preventing organizations from adopting data science and machine learning which can be applied to solve many common business challenges. Collaboration between data scientists, data engineers, business analysts who are working with data (structured and unstructured) from a multitude of sources is an example of one of those barriers.

In addition, there’s a complexity involved when you try to do things with these massive volumes of data. Then add in some cultural aspects, having multiple teams and using consultants, and with all these factors, how do you get that one common theme and common platform where everybody can work and be on the same page? Azure Databricks is one answer.

Here’s an overview of 3 common use cases that we’re beginning to see and how they can benefit your organization:

1. Recommendation Engines – Recommendation Engines are becoming an integral part of applications and software products as mobile apps and other advances in technology continue to change the way users choose and utilize information. Most likely when you’re shopping on any major retail site, they are going to make recommendations to related products based on the products you’ve selected or that you’re looking at.

2. Churn Analysis – Commonly known as customer attrition; basically, it’s when we lose customers. Using Databricks, there are ways to find out what some of the warning signs are behind that. Think about it, if you get ways to correlate the data that leads to a customer leaving your company, then you know that you have a better chance to possibly save that customer.

And we all know that keeping a customer and giving them the service they need or the product they want is significantly less costly than having to acquire new customers.

3. Intrusion Detection – This is needed to monitor networks or systems and activities for malicious activity or policy violations and produce electronic reports to some kind of dashboard or management station or wherever that is captured.

With the combination of streaming and batch technologies tightly integrated with Databricks and the Azure Data Platform, we are getting access to more real-time and static data correlations that are helping to make faster decisions and try to avoid some of these intrusions.

Once we get triggered that there is a problem, we can shut if off very quickly or use automation options to do that as well.

Today I wanted to highlight some of the ways that you can utilize Databricks to help your organization. If you have questions or would like to break down some of these barriers to adopting machine learning and data science for your business, we can help.

We are using all the Azure technologies and talking about them with our customer all the time, as well as deploying real world workload scenarios.

Azure, Database, SQL Server

A Look at Some of Azure SQL Database’s Intelligence Features

Leave a comment

Today I’d like to tell you about some very cool intelligence features within the Azure SQL Database. Azure SQL Database technologies deliver intelligent capabilities through a range of built-in machine learning and adaptive technologies that monitor and manage performance and security for you.

Using telemetry from millions of databases running in Azure over the years, Microsoft has built this capability of training a truly intelligent and autonomous database that gives you the ability to learn and adapt to your workload. This intelligent performance gives you the deeper insight into database performance. Plus, it eliminates the hassle of making ongoing improvements, allowing you to focus more on driving your business and less on “chores”.

Features like query performance insights and automatic tuning continuously monitor database usage and detect disruptive events and then they take steps to improve performance.

Three examples of the intelligent performance that can collectively optimize your memory usage and improve overall query performance are things like:

  • Row mode memory grant feedback – this gives you the ability to expand on batch-mode memory grant feedback by adjusting memory grant sizes for both batch and row mode operators.
  • Approximate query processing – this is designed to provide aggregations across large datasets where responsiveness is more critical than absolute precision, and it will return an approximate value with the focus on performance.
  • Table variable deferred compilation – this improves plan quality and overall performance for queries, referencing table variable by propagating cardinality estimates that are based on actual table variable row counts. In turn, this optimizes your downstream plan operations.

Along all those features, Azure SQL Database intelligent protection allows you to efficiently and productively meet your data’s security and compliance requests by proactively monitoring for potential threats and vulnerabilities. You can flag things such as PII or a cross-scripting attack or something like that. There are detection mechanisms in there that can help you avoid these.

Through features like information protection, vulnerability assessment and threat detection, you can proactively discover and protect sensitive data, as well as uncover potential vulnerability and detect anomaly activities that could indicate a threat to your data.

In short, Microsoft has built these intelligent features over years of machine learning and is applying it to all their Platform as a Service, as well as some of their on-premises, offerings. These are really cool features and we’ve got great response about them and how well they work.

I recommend you give these features a try, but remember, always try them out in your test or dev environments prior to bringing them into production.

Azure, Big Data

What is Azure Data Box Heavy?

Leave a comment

You may have seen my previous Azure Every Day post on Azure Data Box and Azure Data Box Disk. These are a great option for getting smaller workloads, up to 80 terabytes of data, quickly up into Azure. Rather than moving it over the wire, you can send a box and bring it up.

The Data Box Heavy works the same, but you can use much larger amounts of data with up to a petabyte of space.

Let’s review the Data Box process:

  • You order the box through the Azure Portal and specify the region that you’re going to use.
  • Once you receive it, connect it into your network, set up network shares and then you copy your data over. It has fast performance with up to 40 gigabits/second transfer rates.
  • Then you return the box to Microsoft and they will load the data directly into your Azure tenant.
  • Lastly, they will securely erase the disk as per the National Institute of Standards and Technology (NIST) guidelines.

The Data Box Heavy is ideally suited to transfer data sizes larger than 500 terabytes. If you used a Data Box with it’s 80 terabytes, you’d need 5 or 6 of those in place of the Heavy. When you have those larger data sizes, it makes more sense to have it on one machine.

The data movement can be a one time or periodic thing, depending on the use case. So, if you want to do an initial bulk data load, you can move that over and then follow that up with periodic transfers.

Some scenarios or use cases would be:

  • You have a huge amount of data on prem and you want to move it up into Azure – maybe a media library of offline tapes or tape backups for some kind of online library.
  • You’re migrating an entire cabinet – you have a ton of data in there with your virtual machine farm, your SQL Server and applications – over to Azure. You can move that over into your tenant, migrate your virtual machines first, then you can do an incremental restore of data from there.
  • Moving historical data to Azure for doing deeper analysis using Databricks or HD Insight, etc.
  • A scenario where you have a massive amount of data and you want to do the initial bulk load to push it up, then from there you want to do incremental loads of additional data as it gets generated across the wire.
  • You have an organization that’s using IoT or video data with a drone – inspecting rail lines or power lines for instance. They are capturing tremendous amounts of data (video and graphic files can be huge) and they want to be able to move that up in batches. Data Box Heavy would be a great solution to quickly move these up rather than moving the files individually or over the wire.

This is a very cool technology and an exceptional solution for moving data up in a more efficient manner when you have huge, terabyte-scale amounts of data to push to Azure.

Azure, Review, Security

What is Azure Network Watcher?

Leave a comment

Most of us are starting to deploy more and more cloud assets. When you think about how you deploy some assets in Azure, you basically build out a virtual network and you can set that up so it ties in with your on premises network through express route or VPN or you can run it independently in the cloud and have your virtual network set. The question is, how do you monitor and manage that virtual network, like some of the components and how the virtual machines interact? Here’s where Azure Network Watcher comes in.

Azure Network Watcher allows you to monitor, diagnose and gain insight into your network performance between various points in your network infrastructure.

Here’s a breakdown of some of the elements:

1. The Monitoring Element – You can monitor from one endpoint to another with connection monitor to ensure connectivity between 2 points, like a web application and a database for instance. You’ll be alerted with potential issues such as a disconnect between those two services.

It also monitors latency times for evaluation. When you look at those latency times over a period, you’ll know what the average latency is and the max and min. Then you can think about you possibly getting better service in a different Azure region.

2. The Network Performance Monitor – Allows monitoring between Azure and on-premises resources for hybrid scenarios using VPN or express route. It also has some advanced detection to traffic blackholing and routing errors – in other words, some advanced intelligence when it comes to these network issues.

Best of all, as you add more endpoints it will develop a visual diagram of your network with a topology tool which will look like a visio-diagram, showing IP addresses, host names, etc.

3. Diagnostic Tools – From a diagnostic standpoint there are several diagnostic tools that give you better insight into your virtual network by diagnosing possible causes of traffic issues.

IP Flow – Tells you which security rule allowed or denied traffic to or from a virtual machine in your virtual network for further inspection or remediation.

Another tool tests communication for routing rules by letting us add a source and destination IP, then shows the results of that route, again to investigate further or remediate.

The Connection Troubleshooting Tool – Enables you to test a connection between two VMs, FQDN, URI or IDP4 addresses and returns info like the Connection Monitor but only about that point and time latency, not over a span of time.

The Packet Capture Tool – Allows traffic to be captured to and from a virtual machine with some fine-grained filtering to be stored inn Azure storage and further analyzing with network encapture tools like Wire Shark, for instance.

4. Metrics Tools – There are some limitations as to how many resources you can deploy within an Azure network which can be based on subscriptions or regions. The Metric Tool gives you the visibility that you need to understand exactly where you are inside of those limitations. It shows you how many of those resources you’ve deployed and how many are still available that you can deploy – so it helps you set up planning for the future as you deploy more and more resources.

5. Logging – We’ve done some interesting things with log analytics. Log analytics provides the ability to capture data about a bunch of Azure networking components, like network security groups, public IP addresses, load balances, virtual networking and application gateways, to name a few.

All these logs can be captured and stored in Azure storage and further analyzed. Many can be fed into Operations Management Studio (OMS). This gives you a single pane of glass experience when you want to look at your environment at that “50,000-foot level”.

So, as you begin to deploy more and more assets into your Azure environment, this is a helpful service to monitor and manage your virtual network. You get a high-level overview of what that network looks like.