GIG

赴くままに技術を。

AzureAD認証で異なるテナント間のBlobコピー

ドキュメントには特段記載がないと思っていたところ、現在のところサポートされないシナリオでした。

AzCopy v10 を使用して Azure ストレージ アカウント間で BLOB をコピーする | Microsoft Learn

移行元アカウントと移行先アカウントは、同じ Azure AD テナントに属している必要があります。


一応、行うとどんなエラーとなるかメモ。

事前準備

-テナントAでストレージ アカウントを作成 -テナントBでストレージアカウントを作成 -ダミーファイル (1MB) を複数作成し、テナントAのストレージアカウントにアップロード

for /l %n in (1,1,3000) do fsutil file createnew %n 1048576

-操作アカウントに対して、以下のRBACを付与 - テナントAストレージアカウント; ストレージBLOBデータ閲覧者 - テナントBストレージアカウント; ストレージBLOB共同作成者

コピーの実行
PS > azcopy login --tenant-id "aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXXXXX to authenticate.

INFO: Login succeeded.
PS > azcopy login --tenant-id "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXXXXX to authenticate.

INFO: Login succeeded.

azcopy copyを実行すると、どうやらデータコピー元に対して参照操作が認証エラー

PS > azcopy copy "https://<src>.blob.core.windows.net/src/testdata" "https://<dest>.blob.core.windows.net/dest" --recursive
INFO: Scanning...
INFO: Authenticating to destination using Azure AD
INFO: Authenticating to source using Azure AD
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

failed to perform copy command due to error: cannot start job due to error: cannot list files due to reason -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/vsts/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=InvalidAuthenticationInfo) =====
Description=Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:39925b77-701e-0041-5a55-06a0e8000000
Time:2022-12-02T13:56:13.9916736Z, Details:
   AuthenticationErrorDetail: Issuer validation failed. Issuer did not match.
   Code: InvalidAuthenticationInfo
   GET https://<src>.blob.core.windows.net/src?comp=list&delimiter=%2F&include=metadata&prefix=testdata%2F&restype=container&timeout=901
   Authorization: REDACTED
   User-Agent: [AzCopy/10.16.2 Azure-Storage/0.15 (go1.17.9; Windows_NT)]
   X-Ms-Client-Request-Id: [833ede1e-8e87-4dbc-6777-f49b2fdb66e6]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
   Content-Length: [402]
   Content-Type: [application/xml]
   Date: [Fri, 02 Dec 2022 13:56:13 GMT]
   Server: [Microsoft-HTTPAPI/2.0]
   Www-Authenticate: [Bearer authorization_uri=https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/authorize resource_id=https://storage.azure.com]
   X-Ms-Error-Code: [InvalidAuthenticationInfo]
   X-Ms-Request-Id: [39925b77-701e-0041-5a55-06a0e8000000]

下記の開発元からのコメントにあるように、現時点では非サポートの状況

Azcopy Copy - Supported directions missing | Azure Blob (OAuth authentication) -> Azure Blob (OAuth authentication) · Issue #99148 · MicrosoftDocs/azure-docs · GitHub

回避策

コピー元、コピー先のストレージアカウントのどちらかのみに対しAzure AD認証を使うのはOKなので、もう一方は例えばSAS使ってコピーができます。

テナントA (Azure AD) -> テナントB (SAS)

コピー先のコンテナーに対して、SASトークンを発行 (アクセス許可 : 作成)

PS > azcopy login --tenant-id "aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXXXXX to authenticate.

INFO: Login succeeded.
PS > azcopy copy "https://<src>.blob.core.windows.net/src/testdata" "https://<dest>.blob.core.windows.net/dest?sp=c&st=2022-12-03T07:24:55Z&se=2022-12-03T15:24:55Z&spr=https&sv=2021-06-08&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" --recursive
INFO: Scanning...
INFO: Authenticating to source using Azure AD
INFO: Failed to create one or more destination container(s). Your transfers may still succeed if the container already exists.
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job ef855a5f-2cd9-174f-6bc2-d0f8aab4798f has started
Log file is located at: C:\Users\<user>\.azcopy\ef855a5f-2cd9-174f-6bc2-d0f8aab4798f.log

INFO: Could not read destination length. If the destination is write-only, use --check-length=false on the command line.
100.0 %, 3000 Done, 0 Failed, 0 Pending, 0 Skipped, 3000 Total, 2-sec Throughput (Mb/s): 2707.3708


Job ef855a5f-2cd9-174f-6bc2-d0f8aab4798f summary
Elapsed Time (Minutes): 0.1001
Number of File Transfers: 3000
Number of Folder Property Transfers: 0
Total Number of Transfers: 3000
Number of Transfers Completed: 3000
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 3145728000
Final Job Status: Completed

テナントA (SAS) -> テナントB (Azure AD)

コピー元のコンテナーに対して、SASトークンを発行 (アクセス許可 : 読み取り/リスト)

PS > azcopy login --tenant-id "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXXXXX to authenticate.

INFO: Login succeeded.
PS > azcopy copy "https://<src>.blob.core.windows.net/src?sp=rl&st=2022-12-03T07:31:27Z&se=2022-12-03T15:31:27Z&spr=https&sv=2021-06-08&sr=c&sig=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" "https://<dest>.blob.core.windows.net/dest" --recursive
INFO: Scanning...
INFO: Authenticating to destination using Azure AD
INFO: Authentication: If the source and destination accounts are in the same AAD tenant & the user/spn/msi has appropriate permissions on both, the source SAS token is not required and OAuth can be used round-trip.
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job 37be841c-fb7a-f048-603d-33b3c1d44aa2 has started
Log file is located at: C:\Users\<user>\.azcopy\37be841c-fb7a-f048-603d-33b3c1d44aa2.log

INFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.
100.0 %, 3000 Done, 0 Failed, 0 Pending, 0 Skipped, 3000 Total, 2-sec Throughput (Mb/s): 4798.4101


Job 37be841c-fb7a-f048-603d-33b3c1d44aa2 summary
Elapsed Time (Minutes): 0.1005
Number of File Transfers: 3000
Number of Folder Property Transfers: 0
Total Number of Transfers: 3000
Number of Transfers Completed: 3000
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 3145728000
Final Job Status: Completed