Problem
How to get Azure data lake gen 2 container or folder size
Solution
PowerShell AZ Module has to be installed to run below commands
Connect-AzAccount
$myfilesystem = "adventureworks"
$ctx = New-AzStorageContext -StorageAccountName "vcadlsgen2" -UseConnectedAccount
$Files = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $myfilesystem -Recurse `
| Where-Object IsDirectory -eq $false
$Total = $Files | Measure-Object -Property Length -Sum
$Total | Select-Object @{Name = "SizeInBytes"; Expression={$_.Sum}},
@{Name = "SizeInKB"; Expression={$_.Sum/1KB}}
$myfilesystem = "adventureworks"
$mypath = "Address"
$ctx = New-AzStorageContext -StorageAccountName "vcadlsgen2" -UseConnectedAccount
$Files = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $myfilesystem -Path $mypath -Recurse | Where-Object IsDirectory -eq $false
$Total = $Files | Measure-Object -Property Length -Sum
$Total | Select-Object @{Name = "SizeInBytes"; Expression={$_.Sum}},
@{Name = "SizeInKB"; Expression={$_.Sum/1KB}}
Output -
I'm getting error
ReplyDeleteYou can easily run anAzure Data Lake Gen2 instance in a Linux virtual machine on your computer by using Virtual Box, Docker, or any other container technology. To run an instance of the Data Lake Gen2 in a container, you need to install Azure CLI 2.0 and then create a JSON file that describes parameters for the container and the Data Lake Gen2 configuration.
ReplyDeleteHi Vaibhav,
ReplyDeleteApproximately how long would it take to get a response from this script for a 0.5 PB folder?
Big Data is a term used to describe the large amounts of data that are created by the internet and by companies. Thebig data technologies are used to handle big data and increase its value by making sense of the data and extracting meaningful insights that can be used by companies.
ReplyDeleteThanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. abrollcontainer in Germany
ReplyDelete