douglascayers / sfdc-convert-attachments-to-chatter-files Goto Github PK

Tests are failing when no roles are configured for organization. It's necessary to add at least one role.
Perhaps it makes sense to add this information to the readme or updating the test code.
I'm up for doing the pull request for this.

Hi douglas ,

Could you please help me in this why all attachmnets are not converted. i run the job again for those record which was not converted . but still it is not working. is there a limit for particular org .. our org limit is already 200k for 24 hr.but here i am not able to figure it out why it is not working for some of the records. some attachments are not converted ..if i missed something then it should not convert all attachments .

Please help !
Its really Urgent.

Thanks

I can see in the ContentVersion object that the attachments were pushed over to create new content but they are not shared with the parent. My batch job did generate a couple of errors: "Insert failed. First exception on row 0; first error: INSUFFICIENT_ACCESS_OR_READONLY, Invalid sharing type I: [ShareType]" with detail of "(dca_cnvrt_file)". It worked for the custom objects I have, but not Account, Task, Opportunity and Solution. Have you seen this? I'm not a regular chatter user but did follow the steps in the guide to update the options etc.

Hi @douglascayers,

I'm trying to run this in my Partner community however when I create an attachment as a Partner User the debug logs show me: "Insert failed. First exception on row 0; first error: INSUFFICIENT_ACCESS_ON_CROSS_REFERENCE_ENTITY, You do not have the level of access necessary to perform the operation you requested."

The stack trace shows that the error is on insert of ContentVersions (Line 99 of ConvertAttachmentsToFileService).

I have enabled:

• Customer and Partner Portal users must have the “View Content in Portal” permission to query content in libraries where they have access.
• Customer and Partner Portal users can only publish, version, or edit documents if they have a Salesforce CRM Content feature license.

As seen here:
https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_contentversion.htm

The creation of Files from Attachments works correctly as an internal user.

Any Ideas on this that you can encounter?

We have around 300k attachments we have run the job for all records and job completed successfully but some attachment are not converted into files .is there any criteria why it is not working .
is there a way to find those record which are not converted and if we can run the record again.why it is not working ..we might loos the data we can't do the testing for all 300k records.

quick help will really helpfull .

Thanks in advance

Reported by Rick MacGuigan:

Doug, going thru several manual conversions of Notes and Attachments. I'm finding that characters like ', ", <, > , [, ] , and then pasting the text back in from NotePad seems to remove the character error message. Although no guarantee. Not consistent.

As for attachments, are there certain filetypes (i.e. docx,....) that do not convert ? Or does the conversion take longer?

Per Alba Rivas:

In API versions 33.0 and later, you can create and delete ContentDocumentLink objects with a LinkedEntityId of any record type that can be tracked in the feed, even if feed tracking is disabled for that record type. Chatter must be enabled for the organization. I noticed you were explicitly checking in your code that feed tracking it activated for an object in order to do the attachment to file conversion. Maybe you could relax the validation as I think it is not needed anymore?

After converting the attachment with file type pdf, when I try to open it, showing file error message as file damaged.

Is this known issue for pdf file conversion?

Not all objects support files, like EmailMessage and EmailTemplate. The conversion will fail when trying to creating a File shared to those kinds of records.

This feature allows the admin to configure which objects they want to be included in the conversion process rather than all Attachments that exist in the org.

Hi,
I tried the package on a sandboxe and everything works like a charm. Unfortunately, i can't deploy it on production environment due to unit test failures :

ConvertAttachmentsToFilesServiceTest test_inbound_email_and_do_convert System.AssertException: Assertion Failed
Stack Trace: Class.ConvertAttachmentsToFilesServiceTest.test_inbound_email_and_do_convert: line 291, column 1

ConvertAttachmentsToFilesServiceTest test_inbound_email_and_do_not_convert System.AssertException: Assertion Failed: Expected: true, Actual: false
Stack Trace: Class.ConvertAttachmentsToFilesServiceTest.test_inbound_email_and_do_not_convert: line 369, column 1

Am i the only one to experiencing that problem ?

Many thanks in advance for your help

The issue is that the Batch Apex job always has the status of Processing and never completes.

Background:

• In a sandbox, last week I did the installation and successfully completed several "#2 Convert Attachments".

• Then, over the weekend, Summer Release 18 happened.

• And the sandbox was refreshed.

• I reviewed the prerequisites for "Enable Create Audit Fields" and "Email-to_Case". Both are confirmed to be enabled.

• I deployed the package using 'Install for Admins Only'.

• I assigned myself the "Convert Attachments to Files" permission set

• I have ran one single record test that showed still showing "Processing" status after one hour.

• Since this didn't seem right, I uninstalled and deleted the package.

• Reinstalled the package with 'Install for All user'.

• Assigned myself the "Convert Attachments to Files" permission set and then switched to the "Convert Attachments to Files" app

• I have run three single record tests but they are still showing "Processing" status. The first one has been in the job queue for an hour.
  The three single records were:
  0061300001E1HbMAAV
  0061300001C0kRc
  a1813000008mJcpAAE (This is a custom object)

What can I do to get the jobs to get to Completed Status?

Apex Jobs looks like this:

We are currently using Salesforce Quote to Cash (A.K.A. Salesforce CPQ, A.K.A. Steelbrick CPQ). This will generated a quote pdf and save it as an attachment on the SBQQ__RecordJob__c object.

For both the auto conversion process and CPQ to live in the same world, I need to prevent the auto conversion trigger from firing on any records related to any custom objects with the SBQQ__ prefix.

Hi Doug! I am testing your app out in both a full and partial sandbox environment. In both cases, only a subset of the attachments existing in those environments are being converted (5k out of 47k in the full sandbox). Are there common issues that may be leading to this? Our attachments are primarily images and PDFs.

I done with all the setting still clicking on the page i am not redrecting to the page where i should get the option please help me to convert the att

achemnet

Hi Douglas,

I hope your are doing fine so far. I am trying to install you app for converting attachment to files, since it tried installing the one from salesforce labs and it does not let me install it. When I try to install yours into my org, it display the following errors:

This app can't be installed.
There are problems that prevent this package from being installed.

Dependent class is invalid and needs recompilation: Class dca_cnvrt_file.ConvertAttachmentsToFilesBatchable : Dependent class is invalid and needs recompilation: Class dca_cnvrt_file.ConvertAttachmentsToFilesService : Field is not writeable: ContentVersion.CreatedById | ConvertAttachmentsRunOnceController: Dependent class is invalid and needs recompilation: Class dca_cnvrt_file.ConvertAttachmentsToFilesBatchable : Dependent class is invalid and needs recompilation: Class dca_cnvrt_file.ConvertAttachmentsToFilesService : Field is not writeable: ContentVersion.CreatedById.

I thank you before hand for any help you can provide me regarding this matter.

While running the conversion in my full sandbox using version 1.3 of the code, I'm getting a number of errors logs saying "Required fields are missing: [VersionData] Required fields are missing: [VersionData]". I see in issue #25 that it might be related to a required field or validation but I can't figure out where to look for it. Any suggestions?

My organization enable Email to Case. Previously we successfully installed the package but now it asking for EnhancedEmail to be enabled.

How to resolve this
Test failure, method: ConvertAttachmentsToFilesBatchableTest.test_conversion_error -- System.DmlException: Insert failed. First exception on row 0; first error: REQUIRED_FIELD_MISSING, Required fields are missing: [ParentId]: [ParentId] stack Class.ConvertAttachmentsToFilesBatchableTest.test_conversion_error: line 379, column 1

Test failure, method: ConvertAttachmentsToFilesBatchableTest.test_preserve_original_inactive_owner -- System.DmlException: Insert failed. First exception on row 0; first error: INSUFFICIENT_ACCESS_ON_CROSS_REFERENCE_ENTITY, insufficient access rights on cross-reference id: [] stack Class.ConvertAttachmentsToFilesBatchableTest.test_preserve_original_inactive_owner: line 335, column 1

Hi Douglas

We are always getting this error and don't know how to overcome that. Could you please help.

Apex script unhandled exception by user/organization: 005b000000159po/00D5E0000008nQ1
Source organization: 00Db0000000dJIq (null)
Failed to process batch for class 'dca_cnvrt_file.ConvertAttachmentsToFilesBatchable' for job id '7075E00000JRbNZ'

caused by: System.LimitException: Apex heap size too large: 166876089

External entry point
(dca_cnvrt_file)

Since Salesforce enforces a max of 36,000 ContentVersions published in a 24-hour period (can be increased to 50,000 by contacting support), then this poses a problem for orgs with more than 36,000 attachments to convert.

Add new conversion option to limit the max number of attachments the batch job will try to convert in the current day. This will allow admins to throttle how many attachments they convert each day and try to stay under the governor limit.

Cari Aves asked on April 11, 2017:

Does your migration tool handle all the various content types in Salesforce? If not, any plans to support things like Documents?

My reply was:

At the moment, the conversion tool is focused on Attachments but there's no reason why I couldn't add support for Documents or other content types if there's use cases for it. Documents aren't related to records but can be stored in a folder. If we were to convert Documents to Files then I suppose we'd want to look into automating the creation of Folders, which is Beta in Spring '17 release, so that grouped documents stay grouped as files?

Hi Doug,

Thanks for the great tool!

We have a conversion that fails (using the trigger) each time an attachment is added. We have a Salesforce Site which allows the user to add multiple attachments on the visualforce page. When then get added to the created case, they trigger the following conversion error in the log.

Error:
Implementation restriction: ContentDocumentLink requires a filter by a single Id on ContentDocumentId or LinkedEntityId using the equals operator or multiple Id's using the IN operator.

A couple things:

• The user is a Site Guest account
• All the files do not have a Content Type associated with them. But there have been other attachments by regular users which do not have the content types.
• To band-aid this, I run a report several times a day and run a batch of the parentIDs to get the files converted.

Note: I do not have the following setting enabled right now.
Files uploaded to the Attachments related list on records are uploaded as Salesforce Files, not as attachments

Any idea of how to fix this?

Thanks so much!
Anne-Marie

Doug this is a question not a bug report. How does the Attachment Trigger function compare with the Salesforce Files setting "Files uploaded to the Attachments related list on records are uploaded as Salesforce Files, not as attachments". Do they both do the same thing? Should we only use one or the other? We intend to also enable the background job in your App to catch any attachments that miss the automatic conversion (or for old documents).

I've downloaded the conversion tool to my sandbox and have followed the both the per-requisite and Getting Started steps. I added some test attachments to a contact record and have tried running the conversion tool three ways:

• Contact record ID as Attachment Parent Record IDs
• Notes & Attachment object ID as Attachment Parent Record IDs
• No Attachment Parent Record IDs
  My attachments haven't been migrated and there were no error logs created. Any suggestions for how to move forward?

Hi,
I'm getting the following error when trying to install to a partial sandbox using the SandBox link for latest version.
Is there a work around for this?
Thanks!

Anne-Marie

Error Number: 90945798-26744 (-1917862329)
Problem:

1. Implementation restriction: ContentDocumentLink requires a filter by a single Id on ContentDocumentId or LinkedEntityId using the equals operator or multiple Id's using the IN operator.
   ConvertAttachmentsToFilesServiceTest: Implementation restriction: ContentDocumentLink requires a filter by a single Id on ContentDocumentId or LinkedEntityId using the equals operator or multiple Id's using the IN operator.

2. Implementation restriction: ContentDocumentLink requires a filter by a single Id on ContentDocumentId or LinkedEntityId using the equals operator or multiple Id's using the IN operator.
   ConvertAttachmentsToFilesServiceTest: Implementation restriction: ContentDocumentLink requires a filter by a single Id on ContentDocumentId or LinkedEntityId using the equals operator or multiple Id's using the IN operator.

3. Invalid type: ConvertAttachmentsToFilesOptions
   ConvertAttachmentsToFilesTrigger: Invalid type: ConvertAttachmentsToFilesOptions

4. Apex class 'dca_cnvrt_file.ConvertAttachmentsRunOnceController' does not exist
   ConvertAttachmentsFAQPage: Apex class 'dca_cnvrt_file.ConvertAttachmentsRunOnceController' does not exist

5. Apex class 'dca_cnvrt_file.ConvertAttachmentsRunOnceController' does not exist
   ConvertAttachmentsRunOncePage: Apex class 'dca_cnvrt_file.ConvertAttachmentsRunOnceController' does not exist

6. Apex class 'dca_cnvrt_file.ConvertAttachmentsScheduleController' does not exist
   ConvertAttachmentsSchedulePage: Apex class 'dca_cnvrt_file.ConvertAttachmentsScheduleController' does not exist

7. Apex class 'dca_cnvrt_file.ConvertAttachmentsSettingsController' does not exist
   ConvertAttachmentsSettingsPage: Apex class 'dca_cnvrt_file.ConvertAttachmentsSettingsController' does not exist

8. (Convert_Attachments_to_Files) In field: apexClass - no ApexClass named ConvertAttachmentsToFilesServiceTest found
   Convert_Attachments_to_Files: In field: apexClass - no ApexClass named ConvertAttachmentsToFilesServiceTest found

This may be a dumb mistake, but after installing the package and following the instructions, I don't have the Test Conversion option available. The options I have are Setup Conversion, Convert Attachments, Automate Conversion and Troubleshooting. I also don't see any options for test conversion. Can someone please advise on how to find this option?

Add option to the batchable class to delete the original attachments post conversion. This should be careful to only delete attachments that were successfully converted, in the case that the DML option "allOrNone" was set to false.

I want to use attachment as input file on vf page

Hi douglas,

I observed that the app is not able to convert large (more that 200 mb may be ) attachments into files is there any workaround .did i missed something from side . i had around 300k attachments i can see all are concerted except 8k record and those record are having large size.

please help if there is any workaround for larger files.

thanks in advance.

Requested by James Bujold:

If I run the conversion without deletion and then run it again. Will it reconvert everything or will it ignore notes or attachments that were already converted?

Reported by Rick MacGuigan:

Doug, something else I've discover. For attachment conversions if the title of the attachment contains non alpha numeric chars like Meeting Agenda 2016-06-16 agenda.pdf then this will not convert. Nor will it show an error. It would be great to interrogate the title and strip out any of these chars with a space as Meeting Agenda 2016 06 16 agenda pdf which will convert the attachment.

Add option to set the DMLOption for optAllOrNone so that partial success is possible instead of rolling back the entire batch. Developers should be able to set this option on the batchable classes.

https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_methods_system_database_dmloptions.htm

Reported by Martin Vyskocil

I ran a full test of all attachments in my sandbox, and am getting this error in the Apex Jobs log: "ContentPublication Limit exceeded." It shows 1524 batches processed, but 1455 failures. Any idea what would be causing this error? I have 152,000 attachments.

I am trying to convert attachment into files through this app,but this app is only convert into files only when new attachment is uploaded.what about already existing attachment, will it not work for a alreday existing attachment .or i missed something.

Hi Doug, we have a legacy app that allows clients to upload documents (as attachments) to opportunities via a portal. Apparently these anonymous portal users do not have access to Salesforce files (only attachments) so we need to exclude these attachments from being converted (for a couple of opportunity record types). Can the object selection be expanded to add a number of record types to be excluded from the automated (batch) conversion process?

Subject says it all, but the LastModifiedDate on the related list on the parent page has the date of the attachment conversion -- it would be more helpful if it was the latest version date of the attachment.

If CRM Content is not enabled in the org then the conversion fails. It happens in different ways dependent on the settings:

Users can only view the file but cannot edit it, even if the user can edit the attachment's parent record

The following error occurs:

Insert failed. First exception on row 0; first error: INSUFFICIENT_ACCESS_OR_READONLY, You do not have the level of access necessary to perform the operation you requested. Please contact the owner of the record or your administrator if access is necessar

and

Users inherit view or edit access to the file based on their view or edit access to the attachment's parent record

The following error occurs:

Insert failed. First exception on row 0; first error: INSUFFICIENT_ACCESS_OR_READONLY, Invalid sharing type I: [ShareType]

Both of these look like they might be a Salesforce defect, the following code exhibits the same behaviour for me:

Account a = [SELECT Id FROM Account LIMIT 1];
ContentVersion cv = new ContentVersion(versionData = Blob.valueOf('YQ=='), title = 'test', description = 'test', pathOnClient = '/test.txt');
insert cv;
cv = [select ContentDocumentId from ContentVersion where id=:cv.id];
insert new ContentDocumentLink (ContentDocumentId=cv.contentDocumentId, LinkedEntityId=a.Id, ShareType='V');

ie the error messages are the same depending on ShareType when Content disabled, and succeeding with Content enabled. Inserting a ContentDocumentLink in a separate transaction succeeds with Content disabled.

In Summer 17 prerelease orgs the issue is not reproducible, so might disappear in time.

It'd be helpful if the readme could note this, or even better for the package UI to flag it if you are able to detect if Content is enabled.

Hi,

I have a question concerning your tool.
We created a new object and transferred al the notes and attachments from the old opportunities object to a new custom object. But they are only visible in Notes and Attachments and I wanted to use your tool to transfer all these attachements to Files.

But when I am doin this I always receive this error
_Insert failed. First exception on row 0; first error: REQUIRED_FIELD_MISSING, Required fields are missing: [File_Type__c]: [File_Type__c]
Edit Summary: Item 3
_

Do you have any Idea?

Many thanks!

Kind Regards,
Jan

When I am running the batch process multiple times for the same parent records, it appears that the duplicate prevention method (filterAlreadyConvertedAttachmentIds) is not catching already converted files. So I did some digging, trying to replicate what the dup-prevention method does: Retrieve ContentVersion records for already converted attachments.
SELECT original_record_id__c FROM ContentVersion WHERE original_record_id__c='{attachmentId}’
Oddly enough, this query does not return any records for known converted attachment Ids. However, if I include the ContentVersion.Id or the ContentDocumentId in the same query, the record is returned.
SELECT original_record_id__c FROM ContentVersion WHERE original_record_id__c='{attachmentId}’ AND Id='{ContentVersionId}'
The settings are as follows:

• Share the files with all users who have access to the attachment's parent record
• All Users (recommended if communities is enabled)
• Users inherit view or edit access to the file based on their view or edit access to the attachment's parent record (recommended)
• Keep them; I have great plans for the attachments...

While I have attempted to remove files upon completion, that has not worked thus far either, which may be related to the issue. I am running the process as an admin with audit data update privileges.

Hi,

Thanks for posting this! I ran it in my sandbox and it worked great for my standard objects, but attachments were not converted to files for my custom objects. Any idea why that might happen? I enabled convert if feed tracking is disabled and confirmed that the attachments are not private.

Melissa

This app converts attachments to files entirely in Apex and so in bound to the max heap size limits of async apex: 12MB.

Attachments originally could be 5MB each but later increased the limit to 25MB. For orgs with large attachments they can exceed this max heap limit easily and so we need a way to exclude them from the batch job and let the admin convert those separately, perhaps manually with Data Loader, https://help.salesforce.com/articleView?id=loader_content.htm&language=en_US&type=0

This new option would allow the admin to specify the max file size of attachments to consider in the batch job. For example, if using a batch size of 5 then the file size option might be set to 2MB for max potential of 10MB per batch, under the max heap size limit for apex.

One of the main complaints I have received from users since moving from Attachments to Files is that when on the parent account record, you can no longer see the files for child objects like Cases and Opportunities.
I am wondering if you have come across this also, and if there is a solution to mass share up to the parent account as well.

Thanks!
Anne-Marie

As reported by Rick MacGuigan at https://douglascayers.wordpress.com/2015/10/10/salesforce-convert-attachments-to-chatter-files/comment-page-1/#comment-306

Notes won't convert when related to an Account or Opportunity. Notes related to Leads work fine.

Here is a snippet from the debug log:

07:49:22.0 (32469143)|FATAL_ERROR|System.UnexpectedException: Note can't be saved

Class.ConvertNotesToContentNotesService.convert: line 105, column 1
Class.ConvertNotesToContentNotesQueueable.execute: line 41, column 1

Debug Log - Note Can't Be Saved.pdf

I'm getting an error every time I try to create a new note manually that "note contains characters that can't be saved" and thus none of my note conversions have worked.

Maybe it's because I enabled notes AFTER I installed the package - could that be causing the issue?

After the conversion process, the current user ends up being the owner of the new File record and the attachment owner has a role of Collaborator (ShareType = 'C') so they can read/write the file.

It would be ideal that the file owner be the same as the attachment owner.

Reported by Martin Vyskocil and Rick MacGuigan

It's great that we now have a way to know the error message when something goes wrong in the conversion, but it's still difficult to then research what's the underlying cause or to obtain more information to help understand the context.

Please provide more details if possible:

• The Attachment ID that was trying to be converted
• The configuration options that were chosen
• Did the error occur when creating ContentVersion or ContentDocumentLink?
• User ID of the intended owner of the new file and their license type

Reported by Rick MacGuigan

The current way the notes and attachments are converted to chatter notes and files is pure apex, not the api, so the audit fields cannot be set even if the permission is granted to the current user running the conversion code.

Need to investigate if using REST API via callouts from the batchable apex will allow for the conversion to occur and preserve the audit fields.

https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/

https://developer.salesforce.com/docs/atlas.en-us.chatterapi.meta/chatterapi/intro_input.htm#cc_upload_binary_files

https://developer.salesforce.com/docs/atlas.en-us.202.0.api.meta/api/

When I submit a conversion, I get a green bar at the top of the screen saying "Conversion batch job submitted: 7072100000SSy4kAAD", but nothing happens. When I got to the Convert Attachments to Files logs there is nothing there. No errors and nothing converted. I'm not receiving errors anywhere, but nothing seems to be happening. I'm working in a partial sandbox.

The classic Note and Attachment objects have a Private field that denotes whether that record is readable by anyone with read access to the parent record (public) or just the owner and administrators (private).

The current conversion code does not take this into consideration and all converted Notes/Attachments share read access with any user with at least read access to the parent record.

The conversion code needs a new parameter to control this behavior, whether the sharing should respect the Private field or not. If the code respects the Private field then the newly converted Note and File would note explicitly grant sharing to users with read access to the original parent record; it would literally become a private Note or File owned exclusively by the original owner. After conversion, the owner or administrators may grant further sharing as appropriate, even back to the record if desired.