Creating an HDInsight cluster from the Azure portal is very easy. However, sometimes you want all the choices and best practices explained as well as the “how to”. I have created a series of slides with audio recordings to walk you through the process and choices. They are available as sessions 1-8 of “Create HDInsightContinue reading “Create HDInsight Cluster in Azure Portal”
Tag Archives: Azure Blob Store
Master Choosing the Right Project for Hadoop
Hadoop is the hot buzzword of the Big Data world, and many IT people are being told “go create a Hadoop cluster and do some magic”. It’s hard to know where to start or which projects are a good fit. The information available online is sparse, often conflicting, and usually focused on how to solveContinue reading “Master Choosing the Right Project for Hadoop”
Share this:
AzureCopy to the Rescue for an S3 to Azure Blob Copy!
This week I helped a client move files from AWS S3 to Azure Storage blobs. Sounds simple, right? Here’s the tricky part… While there are both Azure and AWS cmdlets for PowerShell, they don’t cooperate. Neither has a cmdlet that accepts credentials from the other and neither accepts arbitrary URLs from outside their own cloud.Continue reading “AzureCopy to the Rescue for an S3 to Azure Blob Copy!”
Share this:
PowerShell works for Amazon AWS S3 too!
Technorati Tags: azure blob store,azure storage,aws,s3,neal analytics,windows azure,powershell More and more we have to work with data in many different locations. This week I got to work with S3 files that were moving to Azure blob storage. I was surprised to find that Amazon has published AWS cmdlets for PowerShell. It took me a littleContinue reading “PowerShell works for Amazon AWS S3 too!”
Share this:
Understanding WASB and Hadoop Storage in Azure
Yesterday we learned Why WASB Makes Hadoop on Azure So Very Cool. Now let’s dive deeper into Windows Azure storage and WASB. I’ll answer some of the common questions I get when people first try to understand how WASB is the same as and different from HDFS. What is HDFS? The Hadoop Distributed File SystemContinue reading “Understanding WASB and Hadoop Storage in Azure”
Share this:
Why WASB Makes Hadoop on Azure So Very Cool
Data. It’s all about the data. We want to make more data driven decisions. We want to keep more data so we can make better decisions. We want that data stored cheaply, easily accessible, and quickly ingested. Hadoop promises to help with all those things. However, when you deal with Hadoop on-premises you have aContinue reading “Why WASB Makes Hadoop on Azure So Very Cool”
Share this:
Azure Maximums and Resource Usage from PowerShell
Technorati Tags: Azure,PowerShell Have you ever struggled to find out how many VM cores, HDInsight cores, storage accounts, or other Azure resources your subscription is set to allow or how many you actually use? Maybe you want to use this information in your automation scripts to avoid trying to create components for which you don’tContinue reading “Azure Maximums and Resource Usage from PowerShell”
Share this:
Access Azure Blob Stores from HDInsight
Small Bites of Big Data Edit Mar 6, 2014: This is no longer necessary for HDInsight – you specify the storage accounts when you create the cluster and the rest happens auto-magically. See http://blogs.msdn.com/b/cindygross/archive/2013/11/25/your-first-hdinsight-cluster-step-by-step.aspx or http://blogs.msdn.com/b/cindygross/archive/2013/12/06/sample-powershell-script-hdinsight-custom-create.aspx. One of the great enhancements in Microsoft’s HDInsight distribution of Hadoop is the ability to store and access HadoopContinue reading “Access Azure Blob Stores from HDInsight”
