if categories are not in use, the function Dmail->dmailer_getBoundaryParts set $this->mailHasContent to false.
This is because den $userCategories contains an empty string.
In line 395 this string is checked by intval which return 0 in this case not -1.
Because intval never return -1, this test ever failes.

The missing image "/typo3/sysext/icons/gfx/ok.png" is requested in CSS for "message-ok" class

It's not possible to fetch content from an internal page, when using HTTP Username & HTTP Password on websites running on https

Reproduce:
Choose Backend Module "Direct Mail" > "Internal Pages" > any Page
Error: An error was encountered. The HTML content could not be fetched.

Affected Versions: 4.1.0, 5.1.0

After merging #60 some codes need to be cleaned up.

direct_mail 5.2.0
Typo3 7.6.19

I can't create a new Scheduler Job for Bounced Emails.
Frame stays empty upen saving. Scheduler job not created.

Hello

Are there any plans or possible timelines concerning making the actual extension compatible with the TYPO3 version 8.6 and higher?

Regards,
Angelo

Hi,
TSConfig setting for importer mod.web_modules.dmail.importer.storage has no effect.
The storage folder given in mod.web_modules.dmail.importer.storage is not given to the function.
change:
$this->makeDropdown('CSV_IMPORT[storage]', $optStorage, $this->indata['storage']) to
$this->makeDropdown('CSV_IMPORT[storage]', $optStorage, $this->indata['pid'])

perhaps change also function makeDropDown to insert checked="checked" in the option tag
public function makeDropdown($name, array $option, $selected, $disableInput='')
{
$opt = array();
foreach ($option as $v) {
if (is_array($v)) {
$opt[] = '<option value="' . htmlspecialchars($v[0]) . '"';
if ($v[0]==$selected) {
$opt[] .=' '.'selected="'.'selected"';
}
$opt[] .= '>'.
htmlspecialchars($v[1]) .
'';
}
}

.......
Kind regards
Harald

Typo3 7.6.22, direct mail 5.2.0
Some warnings occuring, strings not found in array:
Core: Error handler (BE): PHP Warning: Illegal string offset 'mid' in /var/www/virtual/domain.tld/htdocs/typo3conf/ext/direct_mail/Classes/Scheduler/AnalyzeBounceMail.php line 225
Core: Error handler (BE): PHP Warning: Illegal string offset 'rtbl' in /var/www/virtual/domain.tld/htdocs/typo3conf/ext/direct_mail/Classes/Scheduler/AnalyzeBounceMail.php line 224
Core: Error handler (BE): PHP Warning: Illegal string offset 'rid' in /var/www/virtual/domain.tld/htdocs/typo3conf/ext/direct_mail/Classes/Scheduler/AnalyzeBounceMail.php line 223

cleanup locallang XLF files. DE is uploaded to translation.typo3.org see https://forge.typo3.org/issues/76338

The allowed PHP version for direct mail is set currently to 5.3.0. It was still tested with PHP 5.5.
Please raise the version of PHP to 5.5.

Shouldn´t startmarker for Direct Mail boundaries be DMAILER_SECTION_BOUNDARY__BEGIN instead of DMAILER_SECTION_BOUNDARY_?

All 'download as csv' links in 'Mails returned:' do not generate a csv file, but will fill only the textarea field below paragraph is this a wanted behavior?

composer why-not typo3/cms:8.7.0
directmailteam/direct-mail v5.1.1 requires typo3/cms-core (>=7.6,<8.0)

Sometimes it can happen that the simulateUsergroup hook in the TypoScriptFrontendController is called multiple times for the same request. But as the validateAndRemoveAccessToken() method removes the registered access token after the first call, the second one will fail.

As a workaround i commented out the line where the token is removed after it has been successfully validated:

    public static function validateAndRemoveAccessToken($accessToken)
    {
        /* @var \TYPO3\CMS\Core\Registry $registry */
        $registry = GeneralUtility::makeInstance('TYPO3\\CMS\\Core\\Registry');
        $registeredAccessToken = $registry->get('tx_directmail', 'accessToken');
        if (!empty($registeredAccessToken) && $registeredAccessToken === $accessToken) {
            // $registry->remove('tx_directmail', 'accessToken'); <----
            return true;
        } else {
            $registry->remove('tx_directmail', 'accessToken');
            return false;
        }
    }

This works, but may be not the best solution.

Import from: https://forge.typo3.org/issues/62125

Description:

Since TYPO3 Cms uses FAL for image rendering the "old" image rendering wont fit the plantext rendering methods.

Now there are links to images generated that look somewhat like that

http://www.yourdomain.com/uploads/pics/1
http://www.yourdomain.com/uploads/pics/2
http://www.yourdomain.com/uploads/pics/3

My installation specs
I use

• TYPO3 Cms 6.2.4
• directmail 4.0.0
• PHP 5.4.30

Quickfix by omitting images
My "quickfix" so far is to omit images completely.
That can be done by changing lines 89-99 in EXT:direct_mail/pi1/class.tx_directmail_pi1.php

case 'textpic':
    $lines[] = $this->getHeader();
    // if (($CType == 'textpic') && !($this->cObj->data['imageorient']&24))    {
    //     $lines[] = $this->getImages();
        $lines[] = '';
    // }
    $lines[] = $this->breakContent(strip_tags($this->parseBody($this->cObj->data['bodytext'])));
    // if (($CType == 'textpic') && ($this->cObj->data['imageorient']&24))    {
    //     $lines[] = '';
    //     $lines[] = $this->getImages();
    // }
break;
case 'image':
    $lines[] = $this->getHeader();
    // $lines[] = $this->getImages();
break;

My additional info

The same problem happens on TYPO3 7.6.x and latest version of direct_mail. The easier way to debug this is to use the page parameter type=99 but I guess the extension is not using the new way TYPO3 handles file references.

Thanks in advance,
Julian

I try to use AnalyzeBounceMail scheduler task, with your implementation.
You obviously assume that the returning emails are attachments on the bounce emails returning from the server. In my case it is not. The returning emails are embedded in the email body. So I changed the code for the processBounceMail to:

/**
 * Process the bounce mail
 * @param Message $message the message object
 * @return bool true if bounce mail can be parsed, else false
 */
private function processBounceMail($message)
{
    /** @var Readmail $readMail */
    $readMail = GeneralUtility::makeInstance(Readmail::class);
    $midArray = array();
$bouncedMail = $message->getMessageBody();
    // Find mail id
    $midArray = $readMail->find_XTypo3MID($bouncedMail);
    // Extract text content
    $cp = $readMail->analyseReturnError($message->getMessageBody());

    $res = $this->getDatabaseConnection()->exec_SELECTquery(
            'uid,email',
            'sys_dmail_maillog',
            'rid=' . intval($midArray['rid']) . ' AND rtbl="' .
            $this->getDatabaseConnection()->quoteStr($midArray['rtbl'], 'sys_dmail_maillog') . '"' .
            ' AND mid=' . intval($midArray['mid']) . ' AND response_type=0'
     );

     // only write to log table, if we found a corresponding recipient record
     if ($this->getDatabaseConnection()->sql_num_rows($res)) {
            $row = $this->getDatabaseConnection()->sql_fetch_assoc($res);
            $midArray['email'] = $row['email'];
            $insertFields = array(
                'tstamp' => $GLOBALS['EXEC_TIME'],
                'response_type' => -127,
                'mid' => intval($midArray['mid']),
                'rid' => intval($midArray['rid']),
                'email' => $midArray['email'],
                'rtbl' => $midArray['rtbl'],
                'return_content' => serialize($cp),
                'return_code' => intval($cp['reason'])
            );
            return $this->getDatabaseConnection()->exec_INSERTquery('sys_dmail_maillog', $insertFields);
        } else {
            return false;
        }
}

This does the job for me.
Can you explain your adoption, that the returning emails are attachments on the bounce emails?

was wondering about the statistics:
why hast recipients total/sent another count than the "mails sent"?

If mod.web_modules.dmail.simulate_usergroup= is set, no content (plain or html) can be fetched. If the parameter is cleared, content fetching will work.

Page construct:
+ Homepage
+-+ News
| +-+ News SysFolder
|
+-+ SysFolder Newsletter , PageTSConfig with mod...dmail-settings
| +-+ Newsletter Page , Content elements with access settings (or unset)

On the Newsletter Page news records from the News Sysfolder are fetched and displayed. This works well for years.

Now I wanted to protect the Newsletter Page and set access rights to the frontend user group with uid 1. In the PageTSConfig of the SysFolder Newsletter I set mod.web_modules.dmail.simulate_usergroup=1.
This setting results in the error message that no html or plain text content could be fetched.
I tried setting access restrictions to each content element on the Newsletter Page instead but it didn't work either.
It even doesn't work if no access restriction is set at all.

DirectMail version is 5.1.0, TYPO3 version is 7.6.10, PHP is 5.6.25

Hi there,
why is the "delete"email code deactivated?

if ($this->processBounceMail($message)) {
// set delete
//$message->delete();

Cheers
Stefan

Compatibility for TYPO3 7.5 needed - already in development?

I am testing the installation of Direct Mail and I have only succeded in sending the "quickmail" and the "external pages" (other direct mail on TYPO3 6.2.19).

The "internal pages" option always return that there are no HTML or TEXT content.

My site is hosted on a temporary serveur with no domain name... just an IP adress with my user name account. To make life easier for my clients, I've changed their host file (and mine) by adding a temporary domain... it is not possible to change directly on the hosting server.

If I understand correctly, the fetching of the internal page cannot succeed because the script cannot resolve the name VS IP of my temporary site? As the site will be hosted under a real domain name only when finished, I cannot program the newsletter functionnality!

Can you point me to some possible solutions?

Is Direct Mail compatible with fluid and fluid_styled_content?

A million thanks to you!

Eric Thibault

The Exception is "jumpurl: Calculated juHash did not match the submitted juHash"

I'm currently debugging it in Version 4.0.1 and as far as i see the code doesn't change so much.

The problem is the # character and an urlencode() of the character with

$jumpurl = str_replace('#', '%23', $jumpurl);

before line

works for me. Can someone check it in a recent version?

Update
Corrected a typo, now it works for me - Can someone check it in a recent version?

As there is no /tmp directory on windows systems debug output results in an error.

Core: Error handler (BE): PHP Warning: file_put_contents(/tmp/dmail.php): failed to open stream: No such file or directory in C:\wwwroot\...\typo3conf\ext\direct_mail\Classes\DirectMailUtility.php line 1227

When ist the next tag of direct_mail in TER planned? At github there are already some patches included which i can’t find in the newest version (TER).
Thanks!

Field attachment is defined as "not null" in ext_tables.sql. Setting mysql to strict mode requires the value to be set here: https://github.com/kartolo/direct_mail/blob/develop/Classes/DirectMailUtility.php#L950

array(4 items)
   caller => "TYPO3\CMS\Core\Database\DatabaseConnection::exec_INSERTquery" (60 chars)
   ERROR => "Field 'attachment' doesn't have a default value" (47 chars)
   lastBuiltQuery => "INSERT INTO sys_dmail (encoding,charset,priority,sendOptions,includeMedia,fl
      owedFormat,HTMLParams,plainParams,scheduled,scheduled_begin,scheduled_end,us
      e_rdct,long_link_rdct_url,authcode_fieldList,type,pid,from_email,from_name,r
      eplyto_email,replyto_name,return_path,use_domain,long_link_mode,organisation
      ,sys_language_uid,subject,page,tstamp) VALUES ('base64','utf-8','3','1','1',
      '1','','&type=99','0','0','0','1','http://xxx/index.php','
      ','0','45','xxx@xxx','xxx','xxx','xxx','xxx','1','1','xxx','','Newsletter','98','1490318563')" (658 chars)
   debug_backtrace => "{closure}
#21 // TYPO3\CMS\Backend\Http\Application->run
#20 // TYPO3\CMS\Core\Core\Bootstrap->handleRequest
#94 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->handleRequest
#302 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->dispatchModule
#92 // TYPO3\CMS\Core\Http\Dispatcher->dispatch
#201 // call_user_func_array
#41 // DirectMailTeam\DirectMail\Module\Dmail->mainAction
# // DirectMailTeam\DirectMail\Module\Dmail->main
#188 // DirectMailTeam\DirectMail\Module\Dmail->moduleContent
#333 // DirectMailTeam\DirectMail\DirectMailUtility::createDirectMailRecordFromPage
#625 // TYPO3\CMS\Core\DataHandling\DataHandler->process_datamap
#1024 // TYPO3\CMS\Core\DataHandling\DataHandler->insertDB
#1320 // TYPO3\CMS\Core\Database\DatabaseConnection->exec_INSERTquery
#6699 // TYPO3\CMS\Core\Database\DatabaseConnection->debug
#223" (830 chars)

the returnmail.sh, which get and process the bounces, need to be rewrite, since it's not compatible with 7.6

possible implementation:

• using scheduler job to fetch the mail from mailbox
• Mailbox data can be put in each scheduler job

To be able to fetch an email from a mailbox, php-imap extension is a requirement.
Using Fetch PHP library could speed up the implementation. But it still needs php-imap extension.

Direct mail does currently not generate a secure hash for authorization code, so unregistering is not possible. See: bobosch/ods_ajaxmailsubscription#2

The unit tests for "extractHyperLinks" fail. Normally this shouldn't be an issue but for some special cases like:

<a title="Browse to http://google.com for more information" href="http://google.com">Test</a>

The value of the "title" attribute would get jumpurl-encoded instead of the "href" attribute when sending a direct mail newsletter.

If you've got a newsletter page on a password-protected https-page, DirectMail fails to add the configured user and password to the URL.

The method DirectMailUtility::addUserPass only checks for http:// and not https://.

A possible fix would look like this:

if ($user && $pass && preg_match('-^(https?://)(.*)$-', $url, $matches)) {
    $url = $matches[1] . $user . ':' . $pass . '@' . $matches[2];
}

Warning when installing direct_mail Version 5.2.0 that jumpurl is newer than required.

In order to streamline the code, introduce a flash message renderer utility.

We will provide a pull request.

There are currently icons located in folder res/gfx. In order to follow TYPO3 folder recommendation these have to move into the folder Resources/Public/Icons.

References have to changes to new folders.

I have multiple hosting situations where I can't create request on the website domain or localhost directly. One is my local docker development environment and the other is a live website behind a load balancer and a very strict firewall.

What i'm proposing is a global configuration like "overwrite content fetch host" where I can define which host actually works from the php processes perspective. I'd also send the correct Host: header to fake the domain which should actually make the request.

Since this is a very specific problem (which i happen to have multiple times) I volunteer to implement it. I just don't have the time right now and maybe someone else has this issue too so it can be tracked here.

Problem with TYPO3 7.6: When sending the Newsletter with the Direct Mail Wizzard it is not possible to change the properties in step 2 (detailed information). The flexform for the changes opens, but the changes aren't saved.
Im someone would like to change the language Parameter (&L=1), this is not possible in step to. With earlier Versions, this has worked perfectly.
In TYPO3 7.6 before sending a newsletter in an other language, the parameter has to be changed in the direct mail configuration.

IF jumpurl is activated
AND a URL in the newsletter contains markers, f.e. ###USER_uid### (needed for EXT:dmailsubscribe)
AND the user is not found or the marker couldn't be replaced because of some other reason
THEN the JumpurlController creates an invalid URL.

Example:
I have this URL in my newsletter template: http://example.com/subscribe/?a=unsubscribe&c=###SYS_AUTHCODE###&u=###USER_uid###. In the newsletter, the URL is replaced with this: http://domain.com/index.php?id=35071&rid=t_329051&mid=3141&aC=054d92e3&jumpurl=9
If no record with this uid exists or the record is deleted, then the markers are not replaced. And then everything after the first # is no longer a GET-Parameter. And then the EXT:dmailsubscribe SubscriptionController doesn't detect the URL as an unsubscribe URL.

Using %23%23%23 instead of ### in the newsletter template doesn't help, because $jumpurl = htmlspecialchars_decode(urldecode($jumpurl)); is executed even if no user record is found. And it doesn't help for all the already sent out newsletters.

Moving this "decode" line into the if wouldn't help if there are other unreplaced markers, f.e. because there is no matching field in the database. And it would be a breaking change.

Reencoding all remaining ### strings would be a breaking change, too.

I think this whole line could produce a lot more trouble if not only the replaceable markers get decoded.

I know, I can workaround this issue by modifying EXT:dmailsubscribe (I will open an issue there, too), but shouldn't the JumpurlController return valid URLs?

missing default value causes SQL error while fetching the newsletter.

Hi @kartolo,
I have a doubt on how to setup dmailerping.gif

Long story short, the resulting src of the dmailerping.gif of the received newsletter is
http://example.com/index.php?id=74&rid=f_5&mid=19&aC=695171a3&jumpurl=http://example.com/typo3conf/ext/direct_mail/res/gfx/dmailerping.gif

Debugging what happens when this url reaches my server I noticed that, in class JumpurlController and method preprocessRequest, the condition
if (preg_match('#/dmailerping\\.(gif|png)$#', $checkPath) && GeneralUtility::isAllowedAbsPath($checkPath))
is false cause here
$checkPath = /var/www/example.com/htdocs/http://example.com/typo3conf/ext/direct_mail/res/gfx/dmailerping.gif
that indeed is an invalid path on my server and specifically
static::validPathStr($path) = false

I add that at the end the newsletter is marked as viewed in the statistics but there's something that I don't understand about that check.

Am I doing something wrong?

I can't get any infos about retuned Emails in the statistics.
Always empty. We send only HTML Emails.
Is there any additional Code which needs to be in the HTML newsletter?

In one of our customer project we use direct mail. While we made our
server php 5.6 ready we found that direct_mail still uses mysql_num_rows
in the pi1. We fixed that for us and like to share the patch with you.
Maybe you like to apply it.

Index: pi1/class.tx_directmail_pi1.php
===================================================================
--- pi1/class.tx_directmail_pi1.php (Revision 1080)
+++ pi1/class.tx_directmail_pi1.php (Arbeitskopie)
@@ -259,9 +259,11 @@
     */
    function getImagesFromDam(&$images_arr){
        $sql = 'SELECT tx_dam.* FROM tx_dam_mm_ref,tx_dam WHERE tx_dam_mm_ref.tablenames="tt_content" AND tx_dam_mm_ref.ident="tx_damttcontent_files" AND tx_dam_mm_ref.uid_foreign="'.$this->cObj->data['uid'].'" AND tx_dam_mm_ref.uid_local=tx_dam.uid AND tx_dam.deleted=0 ORDER BY sorting_foreign';
-       $res = mysql_query($sql);
-       if(mysql_num_rows($res)>0){
-           while($row = mysql_fetch_assoc($res)){
+       /** @var \TYPO3\CMS\Core\Database\DatabaseConnection $db */
+       $db = $GLOBALS['TYPO3_DB'];
+       $res = $db->sql_query($sql);
+       if($db->sql_num_rows($res)>0){
+           while($row = $db->sql_fetch_assoc($res)){
                $images_arr[] = $this->siteUrl.$row['file_path'].$row['file_name'];
            }
        }

I would like to use direct mail with categories width TYPO3 v8.7. I have completed the steps of the instruction. One step is to add the file class.tx_directmail_container.php, but this file does not exist?
When I create a tt_address record, I get a database error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')) AND ( 1=1) AND (pages.uid = sys_dmail_category.pid) AND ((`sys_dmail_' at line 1.

There is no way to select the direct mail category at tt_address.

Greetings
Niklas

Hi,

I've extended the fe_user table with two more fields for e-mail-addresses.
How can direct_mail use this two additional fields?

With the mailHeadersHook I cannot modify/extend/replace the recipient by setting the appropriate header "To".
Is there any way to configure additional fields direct_mail can looking for or to override the recipient at any hook?

My environment:

• TYPO3 v7.6.10
• direct_mail v5.2.0

Greetings,
FireLizard

Setting value for mod.web_modules.dmail.testmail has no effect.

Newest version 5.2.2 of direct mail is not available on packagist or typo3-ter. Update via composer not possible.

Typo3 7.6.20 + Direct mail 5.1.1
I use Language module.
Files are downloaded and translates but aren't use in backend module and still in English.

Thanks

DirectMailTeam\DirectMail\DirectMailUtility::getFEgroupSubgroups() is called statically inside DirectMailTeam\DirectMail\DirectMailUtility::getStaticIdList() (line 374, @3dc8bd4) but is not declared as static. Newer versions of PHP will throw a Runtime Deprecation Notice facing this.

The called method, however, never refers to the current instance and therefore could be declared as static safely.

This will avoid the notice to be thrown.

The layout of the backend module should be closer to the new style.

• The accordion should look like the table head in list view
• The input fields should renderes with the standard styles

Because of the server security policy in my company it's only allowed to fetch pages when giving some kind of user-agent at the request. For this reason I patched the code for direct_mail locally.

Perhaps it makes sense to take this code into master?

In file Classes/DirectMailUtility.php line 1183:

before:
$mailContent = GeneralUtility::getURL(self::addUserPass($plainTextUrl, $params));

after, with user-agent:
$mailContent = GeneralUtility::getURL(self::addUserPass($plainTextUrl, $params), 0, array('User-Agent: Direct Mail'));

In case of multiple direct mail directories for multiple editors, it may happen that a newsletter is temporarily "blocked" from starting because another newsletter is currently being sent. Since we may have large mailing being sent, and the queue is only processed by small chunks, it may literally last hours before another newsletter starts being processed.

In such case, it would be very useful that the editors get a hint that their newsletter is currently pending for another newsletter before starting to be processed. Otherwise they may think that the system is down.