How to store extended file attributes in iCloud

AppleInsider might earn an affiliate fee on purchases made by hyperlinks on our website.

Apple’s macOS and another working programs retailer descriptive details about information referred to as metadata on storage volumes in prolonged attributes (xattr). Prolonged attributes comprise extra details about information utilized by working programs.

By storing and retrieving prolonged attributes, working programs can modify data about information with out altering file contents instantly.

For instance in macOS Finder, when you choose a file and press Command-I on the keyboard, Finder opens a Get Data window and reads among the file’s prolonged attributes and shows them.

Some prolonged attributes are created and managed by macOS itself, however purposes can create their very own prolonged attributes and retailer them as effectively. It’s as much as every app to determine the way it desires to handle prolonged attributes.

Some filesystems do not help prolonged attributes in any respect, similar to NFS. In case you copy information to a filesystem which does not help prolonged attributes, these attributes are misplaced eternally.

Commonest fashionable filesystems do help prolonged attributes, however might handle the info in numerous methods. On most working programs, prolonged attributes are usually not saved with information themselves, however in quantity metadata catalogs, which shops the extra attributes in lists or databases, for every file.

When a file will get copied, the metadata will get copied together with it and inserted into the vacation spot quantity’s metadata catalogs to protect the extra attributes. Utilizing this conduct working programs transfer file metadata from one quantity to a different together with information themselves throughout copies.

Apple publishes restricted, however not all, developer data in regards to the macOS metadata system and the way it works. There at present isn’t any user-level Apple documentation for managing metadata on macOS or iOS.

We do know that in macOS, prolonged attributes and metadata are organized below reverse area notation – very like app bundle names and firm group are. Just a few Apple reverse domains used for metadata embody:

1. com.apple.FinderInfo
2. com.apple.metadata
3. com.apple.TextEncoding

All prolonged attributes are usually not required in each file – some information could also be lacking xattrs from sure reverse domains fully.

When Apple launched iCloud Drive as a means of copying and storing information within the cloud, customers started to note that some, however not all file metadata was misplaced. Worse, this conduct turned inconsistent throughout totally different variations of macOS, someday across the launch of Mac OS X Mojave.

Customers observed that some — however not all — prolonged attributes get stripped from information when moved into iCloud after which copied again out.

At minimal we all know that every one prolonged attributes from the com.apple.FinderInfo and all user-added metadata together with Get Data feedback within the com.apple.metadata area get deleted when information are moved into iCloud.

Prolonged attributes similar to these within the com.apple.TextEncoding area and sure different domains are preserved. In a means this is sensible since each app and even the Finder should understand how textual content knowledge inside every file is encoded to be able to show it accurately.

Then again it might not make sense to Apple for iCloud to protect some attributes similar to com.apple.FinderInfo since that knowledge could also be distinctive to Finder show preferences on every person’s Mac.

Nonetheless, stripping information’ Get Data person feedback is clearly knowledge loss. Worse, there have been experiences of iCloud Drive stripping third-party prolonged attributes, from information throughout totally different variations of macOS when accessing the identical information is in iCloud. That is clearly unhealthy.

Thus far, Apple hasn’t publicly addressed or commented on this, so we do not know if that is supposed conduct or an unaddressed concern.

What Apple has accomplished since someday round 2013 is to quietly add kind of a developer hack to file metadata referred to as prolonged attribute tags.

Apple’s xattrs tags system consists of appending considered one of 4 characters (C, P, N, or S) to the tip of every xattr preceded by the #’ character. Since #’ is likely one of the characters forbidden in xattr names, flags will get ignored by most utility code that handles xattrs.

Every of the 4 flag characters corresponds to an xattr metadata preservation degree in Apple’s programming interfaces. We can’t get into these particulars right here, however suffice it to say the P, N, and S flags correspond, roughly to those attribute preservation ranges:

1. C – Protect when file knowledge adjustments.
2. P – Protect throughout copying.
3. N – All the time strip xattrs – even when copying.
4. S – All the time protect xattrs when syncing.

Therefore, when copying information into iCloud Drive, any file xattrs which have the “#S” flag appended to the tip might be preserved.

Presently there is no such thing as a simple technique to make these modifications to information with out utilizing the Terminal and understanding precisely what adjustments must be made.

And we should always warn you that altering prolonged attributes could be dangerous enterprise. Any errors in xattrs can change the way in which information are displayed in Finder, Get Data home windows, file creation and modification dates, and might even in some circumstances render information broken or unusable.

We all know for sure that as of macOS Ventura, Highlight indexing is aware of learn how to deal with extra xattr flags with out confusion or knowledge loss, however the Finder’s Get Data home windows do not at all times. Often, including xattr flags to xattrs which the Finder’s Get Data makes use of can render information unreadable and probably even unopenable from the Finder.

Additionally observe that the Finder will at all times honor any xattr flags throughout copies – for instance if the ‘#N’ flag exists in any xattrs in any file throughout a native Finder copy, these attributes will at all times be stripped from the copied information in all circumstances.

Then again, the Terminal file copy command (cp) will at all times protect all xattr flags in all circumstances – copying all xattrs on all copied information it doesn’t matter what.

You may view the reasonably sparse developer data on xattr flags in Terminal by typing:

man xattr_name_with_flags and urgent Return.

However except you are a developer, the data supplied could also be obscure because it’s principally Standard C Library programming data included within the C header file xattr_flags.h

You may view and alter information’ xattrs within the macOS Terminal utilizing the xattr command, however be extraordinarily cautious when doing so.

You may as well use customary Terminal listing listings to see whether or not a given file has xattrs or not. To take action, in Terminal, navigate to the listing whose information you need to view utilizing the ‘cd’ command, then kind:

You may see a list of the listing, much like this:

On the left facet of the listing itemizing are every file’s UNIX permissions. File permissions which finish in “@” or “+” point out information which have prolonged attributes hooked up to them.

To see which reverse domains every file’s attributes belong to, kind:

The itemizing would be the similar as utilizing “ls -l”, besides that any xattr reverse domains utilized in information may also be listed:

The xattr UNIX command is a technique to view and alter xattrs on information, however as we talked about above, be extraordinarily cautious when utilizing it. The command is highly effective and might destroy file usability simply.

You may view all xattr command choices in Terminal by typing:

man xattr and urgent Return.

Press Management-Z to exit the person system.

In abstract, there’s no easy and simple technique to take care of xattrs when copying information to and from iCloud.

So long as a given xattr’s flag is about to ‘#S’ when syncing to or from iCloud, that xattr will at all times be preserved.

Apple ought to repair this within the macOS Finder and in iOS, however doing so presents an issue: it could require including the ‘#S’ tag to all file xattrs when syncing, and doing so would probably break at the least some third occasion apps which learn and write xattrs however do not but know learn how to deal with tags.

Developer Howard Oakley has written a number of apps and instruments to assist take care of xattrs – together with an editor referred to as xattred (rhymes with ‘scattered’), and the command line instrument cmpxat which compares and shows all xattr variations between two information.

He is additionally written Cirrus – an iCloud browser instrument – and his blog has a number of discussions on the iCloud xattr concern.

He additionally has two very fascinating virtualization apps, Viable and Liviable which run macOS and Linux digital machines on Apple Silicon Macs.

In case you can learn C code, Apple’s open supply file xattr_flags.c reveals you the internal workings of the xattr tag code Apple added in 2013.

Hopefully within the close to future Apple will arrive at an answer which solves the entire above iCloud issues and seamlessly preserves xattr metadata throughout all iCloud file syncing.