Add support for associations about alfresco-bulk-filesystem-import

[GoogleCodeExporter]

Minor clarification: by "associations" I mean peer-to-peer (graph) associations 
as well as non-standard (non cm:contains) parent/child (hierarchical) 
associations.

Standard (cm:contains) parent/child associations are already supported by the 
bulk filesystem tool via the folder hierarchy of the source directory.

Original comment by [email protected] on 4 Dec 2010 at 2:03

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Note: associations can currently be loaded by specifying the NodeRef of the 
target, but this is problematic for 2 reasons:
1. NodeRefs are a fragile way of specifying the target of an association
2. if an association is being created between two files that are part of the 
same import, the user has no way of knowing what the NodeRef of the target file 
will be (since it doesn't have a NodeRef until such time as the BFSIT loads 
it).  Chicken, meet egg!

The solution I'm envisaging involves allowing associations to be specified some 
other way (e.g. using a path), and then having the BFSIT figure out what the 
NodeRef is based on that path.  This also explains why the process will have to 
be two pass - it's possible that file A has an association to file B, but gets 
imported first (meaning that file B doesn't have a NodeRef yet).  The solution 
is to load all the files first, then go back and "decorate" them with their 
associations (with the obvious performance optimisation that if file B does 
happen to have been loaded already, the association from file A would be 
created immediately, in the first pass).

Original comment by [email protected] on 28 Aug 2011 at 5:01

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

This concept also extends to properties of type d:noderef - conceptually they 
are equivalent to "true" associations.

This is relevant because tags are of type d:category, which is actually 
implemented in exactly the same way as d:noderef (they both share the same 
underlying Java implementation class), and tags are a commonly used in imports.

Original comment by [email protected] on 7 Oct 2011 at 5:26

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Note to self: mixing up reads (SELECT) and writes (INSERT / UPDATE / DELETE) in 
the same database transaction can have a significant impact on performance due 
to blocking and the potential for deadlocks.

For that reason the import tool attempts to only perform INSERTs in the "write" 
transactions it performs, since INSERTs cannot block each other (and hence also 
can't deadlock).

The risk inherent in this issue is that it involves reads (SELECT) in order to 
convert a path reference to a NodeRef, so any solution needs to think very 
carefully about how that's done.  It may be possible to use a short, 
independent, R/O transaction for this step, for example.

Of course this still isn't ideal, since reads (for any reason) tend to be seeky 
in nature, hurting the database's I/O performance even if no blocking occurs.

Original comment by [email protected] on 26 Oct 2012 at 4:58

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Migrated to https://github.com/pmonks/alfresco-bulk-import/issues/16

Original comment by [email protected] on 12 Dec 2013 at 9:25

• Changed state: Migrated
• Added labels: ****
• Removed labels: ****