Get Azure Data Lake Gen2 Container, Folder Size

Problem

How to get Azure data lake gen 2 container or folder size 

Solution

PowerShell AZ Module has to be installed to run below commands

Run below code to connect to your Azure account

Connect-AzAccount

To get the container total size, specify the container (file system) name in the variable at first line and run the code

$myfilesystem = "adventureworks"

$ctx = New-AzStorageContext -StorageAccountName "vcadlsgen2" -UseConnectedAccount

$Files = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $myfilesystem -Recurse `

            | Where-Object IsDirectory -eq $false

$Total = $Files | Measure-Object -Property Length -Sum

$Total | Select-Object @{Name = "SizeInBytes"; Expression={$_.Sum}},

                       @{Name = "SizeInKB"; Expression={$_.Sum/1KB}}  

Output - 

To get the folder size, specify the container (file system) name as well as folder path in the variable

$myfilesystem = "adventureworks"

$mypath = "Address"

$ctx = New-AzStorageContext -StorageAccountName "vcadlsgen2" -UseConnectedAccount

$Files = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $myfilesystem -Path $mypath -Recurse | Where-Object IsDirectory -eq $false

$Total = $Files | Measure-Object -Property Length -Sum

$Total | Select-Object @{Name = "SizeInBytes"; Expression={$_.Sum}},

                       @{Name = "SizeInKB"; Expression={$_.Sum/1KB}} 

Output -

Get Azure Data Lake Gen1 Folder Size and files count

Problem

How to get Azure data lake gen 1 folder size and files count in it

Solution

Option 1 - Azure Storage explorer

1. Install Azure storage explorer, log in to your Azure subscription and open the target folder.

2.  Click on Folder Statistics button on top and you should receive the number of files, subfolders and total folder size in bytes

Option 2 - PowerShell

1. Install Az.Datalakestore Module in order to run Azure data lake gen1 related PowerShell commands using below script

Install-Module -Name Az.Datalakestore -AllowClobber -Force

2. Edit below script to enter your Service Principal Id & Secret and run

$TenantId = "<<YourTenantId>>"

$ServicePrincipalId = "<<YourServicePrincipalId>>"

$ServicePrincipalKey = "<<ServicePrincipalKey>>"

$SecurePassword = ConvertTo-SecureString $ServicePrincipalKey -AsPlainText -Force

$Credential = New-Object System.Management.Automation.PSCredential ( $ServicePrincipalId, $SecurePassword)

Login-AzAccount -ServicePrincipal  -TenantId $TenantId -Credential $Credential

$dataLakeStore = "YourADLSGen1AccountName"

$inputPath = "/YourFolder"

Get-AzDataLakeStoreChildItemSummary -Account $dataLakeStore -Path $inputPath

3. It should return details like below