Limiting SMTP sending to smarthost

This is the Mailtraq Peer Support forum. Get assistance using and managing Mailtraq, and help others solve problems too.

Limiting SMTP sending to smarthost

Postby apurdam » Mon Jun 20th, 2016 12:46am

Mailtraq Version: 2.17.7.3560
I must confess, I moved the two business' email hosting systems from MTQ to Google Apps for Work. (I still use MTQ for outbound from one practice, see below).
It's a very notable increase in cost, but we needed better handling of the size of emails, I needed to change my backup-process and be able to share administration with less tech-savvy people, and the mail database was getting too big to backup off-site conveniently, and I really wanted a better webmail interface.

However, Gmail has really strict limits even with smtp-connected outgoing mail (it's even worse if we connect directly... but that's not allowed by gmail anyway).
Our in-clinic practice management system can be set to x emails per y seconds, but has operational constraints that I'd like to shift off the reception desk (ie I don't want them accidently aborting a 300 user mail-out with an inadvertent mouse-click).

To that end, can I set up the Outbox in Mailtraq so that it only sends - say - 1 email out to the smarthost every 15 seconds?

The throttling will ease once our first payment goes through, but there'll still be a limit, only a larger one. This has really thrown our client-reminder system in the air, as we are getting so many bounces from gmail right now.

Thanks for any help.
Cheers,
Andrew (still paid up with MTQ, btw, despite my apparent lack of loyalty)
User avatar
apurdam
Expert User
 
Posts: 495
Joined: Thu Apr 19th, 2007 10:57pm
Location: Canberra

Re: Limiting SMTP sending to smarthost

Postby Martin Clayton » Tue Jun 21st, 2016 12:10pm

apurdam wrote:can I set up the Outbox in Mailtraq so that it only sends - say - 1 email out to the smarthost every 15 seconds?

Much might depend on the nature of the inbound traffic flow but first thoughts...

I'm guessing that a Custom Routing Table entry with "Limit delivery to 'n' recipients per hour" wont have sufficient granularity -- i.e., it'll go flat-out until it hits the limit, then wait for the hour to elapse. If that assumption is wrong then you're good to go.

Mailbox Accounting incurs a similar granularity issue and runs into the problem of how the local sender responds to any deferral or rejection - admin best avoided. So, I'm mainly thinking accept everything and slow release by script.

Possible routes...
  • Message Filter to control InQueue-OutQueue flow (webmail may bypass the filter)
  • Send to a local slot and Forward when ready
  • Subvert the outbox moderation functionality with timed auto-approval (could be dead easy or tricky)
  • Dynamically adjust "Limit delivery to 'n' recipients per hour" (could be tricky)
  • Or, write messages to mail/pending suitably slowly
User avatar
Martin Clayton
Expert User
 
Posts: 529
Joined: Sat Jan 15th, 2005 8:20am
Location: London, UK

Re: Limiting SMTP sending to smarthost

Postby apurdam » Tue Jun 21st, 2016 7:36pm

Hi Martin,
Thanks heaps for your reply.

I was hoping a Custom Routing Table would do it, but I can't find anything that says "Limit delivery to ...", which would have been perfect (google's granularity is per hour anyway). Is there a rock somewhere that I need to look under, or am I being thick and it's right in front of me?
I'm using 2.17.7.3560
Reading the Custom Routing Table documentation, it struck me as ironic that a feature developed for speeding up delivery was hoping to be co-opted for slowing it down! :-)

Thinking about the possible routes you suggested:
  • Message Filter might work, as I can simply sleep a short while, look for a message, if I find one, sleep a long while, then pass it on. I'm assuming the Message Filter wouldn't be multi-threaded by MailTraq, would it? More programming than what I had hoped...
  • Send to a local slot and Forward when ready. Can't think how to automate that.
  • Subvert the outbox moderation Again, I'd need to write more code than I want
  • Limit delivery to.... I wish!. That would fix it instantly if I could find the setting.
  • Write message suitably slowly This is the process that I'm using, but the setting on the mail generator that we're using means that it invokes the delay before sending, so if sending a single email, you have to wait - say - fifteen second before it says Sent, which disrupts work flow for our receptionists a lot. Have spoken to their support guys to see what can be done.
Currently the last solution is what I'm using, with the subsequent annoyance.
Thanks again for your thoughts. If you know where "Limit delivery to 'n' recipients per hour" is, I'll be exceedingly grateful!
User avatar
apurdam
Expert User
 
Posts: 495
Joined: Thu Apr 19th, 2007 10:57pm
Location: Canberra

Re: Limiting SMTP sending to smarthost

Postby Martin Clayton » Wed Jun 22nd, 2016 6:24am

apurdam wrote:I can't find anything that says "Limit delivery to ...", which would have been perfect (google's granularity is per hour anyway). Is there a rock somewhere that I need to look under, or am I being thick and it's right in front of me?

I can't answer that ... but I will. ;) I'm seeing it with multi-domain beta 3566 at Outbox Properties > Delivery > Use Custom Routing Table > Edit Table (addres spec '*', no mx lookup, host = gmail...).

(I'm not sure if Outbox Properties > Options > Enable advanced Outbound Router affects rate-limiting).

apurdam wrote:Reading the Custom Routing Table documentation, it struck me as ironic that a feature developed for speeding up delivery was hoping to be co-opted for slowing it down! :-)

No, it still fits -- you're trying to slow it down to speed it up. ;)

apurdam wrote:I'm assuming the Message Filter wouldn't be multi-threaded by MailTraq, would it?

No, Mailtraq just picks up whatever's in the OutQueue directory, file-by-file, so control comes from timing what you send to the OutQueue.

apurdam wrote:Send to a local slot and Forward when ready. Can't think how to automate that.

A scheduled script could scan the mailslot, track outbound rate, pluck out message+recipient and send as appropriate.

apurdam wrote:Write message suitably slowly This is the process that I'm using, but the setting on the mail generator that we're using means that it invokes the delay before sending, so if sending a single email, you have to wait - say - fifteen second before it says Sent, which disrupts work flow for our receptionists a lot. Have spoken to their support guys to see what can be done.

I'm more than sure that can be improved. :)
You do not have the required permissions to view the files attached to this post.
User avatar
Martin Clayton
Expert User
 
Posts: 529
Joined: Sat Jan 15th, 2005 8:20am
Location: London, UK

Re: Limiting SMTP sending to smarthost

Postby apurdam » Thu Jun 23rd, 2016 2:51am

Thanks again for your help Martin.
The rate limiting must have come in with the betas since the last major release.
I can't find where to download 3566, but I think the one hour granularity will still be a problem, anyway, as gmail-hosting has 10 minute limits as well. (sigh)

I think I'll see how we go with the current arrangements, as we should now be on the higher limits.
Thanks for all of those ideas, though. I'll revisit them as needed if I have time to implement them! :lol:
Cheers,
A
User avatar
apurdam
Expert User
 
Posts: 495
Joined: Thu Apr 19th, 2007 10:57pm
Location: Canberra

Re: Limiting SMTP sending to smarthost

Postby Martin Clayton » Thu Jun 23rd, 2016 4:57am

apurdam wrote:The rate limiting must have come in with the betas since the last major release.

Ah, I didn't realise it was that recent. (In case it's related to licence, I'm on "Mailtraq Professional with proxy (10 users)").

apurdam wrote:I can't find where to download 3566

This is a multi-domain build -- you'd probably have to rewrite the configuration to run these 2.18 betas.

Pointing wget --spider at resources.mailtraq.com/files/MailtraqWixSetup[????].exe shows "[application/octet-stream]" for;

Code: Select all
3560 : Length: 37022648 (35M)
3562 : Length: 37022648 (35M)
3564 : Length: 37247928 (36M)
3566 : Length: 37247928 (36M) (multi-domain)
3568 : Length: 37202872 (35M)
3570 : Length: 37202872 (35M)
3572 : Length: 37202872 (35M)
3574 : Length: 37247928 (36M)
3576 : Length: 37202872 (35M)
3578 : Length: 37247928 (36M)
3580 : Length: 37247928 (36M)
3582 : Length: 37202872 (35M)
3584 : Length: 37202872 (35M)
3586 : Length: 37252024 (36M)
3588 : Length: 37202872 (35M)
3590 : Length: 37252024 (36M) (multi-domain)
3592 : Length: 37168128 (35M)
3594 : Length: 37209088 (35M) (multi-domain [marked 'broken'])
3596 : Length: 37257784 (36M) (multi-domain)
3598 : Length: 37208632 (35M)
3600 : Length: 37285056 (36M) (2.17)


Unchartered waters but if you want to try it, check the file version for "2.17" before installing - 2.18 is multi-domain. I'm still on the 3566 beta because IMAP auth stopped working for an iPhone in later multi-domain builds.

apurdam wrote:but I think the one hour granularity will still be a problem, anyway, as gmail-hosting has 10 minute limits as well. (sigh)

My assumption about granularity / burst-mode may be wrong so it could still be worth a try. Otherwise it's looking like scripting or a new smarthost and/or CRM interface.
User avatar
Martin Clayton
Expert User
 
Posts: 529
Joined: Sat Jan 15th, 2005 8:20am
Location: London, UK

Re: Limiting SMTP sending to smarthost

Postby apurdam » Thu Jun 23rd, 2016 8:30pm

Thanks again for all your input.
Cheers,
A
User avatar
apurdam
Expert User
 
Posts: 495
Joined: Thu Apr 19th, 2007 10:57pm
Location: Canberra

Re: Limiting SMTP sending to smarthost

Postby apurdam » Fri Jun 24th, 2016 3:07am

Well, that's disappointing. I just upgraded to 2.17.7.3600, and it still doesn't have the rate setting. :(
StillNoSendRateLimiting3600-single-domain.jpg

A shame. I expect it would have done the trick. My license is only for 2.17.

I'm spending waaay too much time on this! :shock:
You do not have the required permissions to view the files attached to this post.
User avatar
apurdam
Expert User
 
Posts: 495
Joined: Thu Apr 19th, 2007 10:57pm
Location: Canberra

Re: Limiting SMTP sending to smarthost

Postby Martin Clayton » Fri Jun 24th, 2016 6:24am

apurdam wrote:StillNoSendRateLimiting3600-single-domain.jpg

Hmm, there's space, I wonder if it's edition specific ("Pro" here).

Just to prolong the agony ;) you might be able to work the Outbox with a suitably twisted combination of;

  • Options > Maximum number of parallel delivery connections
  • Options > Delay connections to smarthosts by 'n' seconds
  • SMTP > Limit Recipients per Message to 'n'
User avatar
Martin Clayton
Expert User
 
Posts: 529
Joined: Sat Jan 15th, 2005 8:20am
Location: London, UK


Return to Mailtraq Support

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron