"Negative Delta" file approach
Hi everyone
For long term discussion, we need to formulate our proposal for the "Negative Delta" file approach, that we all agreed in the previous meeting was preferable to simply just deleting invalid existing content. The aim is to provide a valid methodology by which we can remove technical/content anomalies that would otherwise have been permanent invalid features, without providing any benefit to the terminology.
The proposal would need to enable to cleansing of these issues within the existing RF2 structure, whilst not losing the complete audit trail (as we do with deletion), providing a machine readable way to trace the history going forwards.
This will likely have a reasonable impact on the users in the community, and so this change will have to be thoroughly disseminated out and socialised with all members, vendors and affiliates, so let's start putting the proposal together now with the aim to finalise it in October, and socialise it in time for inclusion in the July 2018 International Edition.
Please submit all of your thoughts on pros/cons, and proposals of how best this will work (including any technical restrictions, etc).
Thanks very much!
Andrew
Comments
Hi,
I think that we need some kind of document with rules about when it is allowed to use negative deltas and when it is not allowed to do it. (I also think that only rare cases should be allowed.)
Regards
Mikael
I agree with @Mikael Nyström (Unlicensed) that there should be rules of at very least very strong guidance around the use of this feature. However I think it is the sort of "break glass" feature needed to deal with some of the anomalous circumstances we inevitably get ourselves in.
I also think @Peter Williams is right, I'm not sure that another active status is the right thing to do - it would be a breaking change.
Personally I'd always thought of this as a parallel structure of release files within the zip which contained the set of "deleted" rows. If you merge this parallel structure of files with the release files you get back to the original (anomalous) state. So a structure mirroring the release directory structure and files, but under a new top level directory called..."deleted"? (or something like that)
I think it would be prudent to also indicate what the files are using the file naming. We prepend "x" to indicate a test version of a file and "z" to indicate an archived artefact, perhaps we need a new letter to prepend with for these? "d" for deleted? That way if a file is separated from the directory structure it is still obvious what it is.
A prepended character is perhaps also a bit too subtle for humans, but I think it is important to trip up automated import routines - I know what ours wouldn't pick up a file starting with a prepended "d" for example. You might want to add something more explicit and obvious in the file naming for humans too. But make it so the file name is machinable to get the release file name the deleted rows originally came from.
Other than that I'd make the content of those files the exact unadulterated rows that were removed from the release files. I think one set of files representing the "full" set of rows removed is probably all that's needed - so that set of rows would need to be applied to the "full" release format. A snapshot of these rows doesn't make much sense.
A delta form (what new "deleted" rows were added in this release) may be useful as a trigger for a release consumer to do something. They could safely ignore this deleted row set until it changes - the delta could make it explicit that it has changed and in what way. Otherwise a consumer would have to compare the full set of deleted rows from this release to the last release to determine if anything had changed.
Anyway, that's my thoughts. To make it happen I think we need
an addition to the bundle structure
file naming convention additions
rules, guidance, documentation
a communication plan and specific communications
In terms of a deletion mechanism, I considered using some other symbol for active state, but that would a) cause issues for existing software and b) not allow fine grained distinction to be made between the the effectiveTime of the row being deleted and the effective time of the deletion itself. I think the solution should involve using a new file type which is then optional for inclusion and doesn't require any modification of existing systems.