Skip to content

Mailgun integration#83

Open
zemccartney wants to merge 11 commits intomasterfrom
mailgun-integration
Open

Mailgun integration#83
zemccartney wants to merge 11 commits intomasterfrom
mailgun-integration

Conversation

@zemccartney
Copy link
Copy Markdown

@nelsonic heyo! got sending with Mailgun working, tests passing, all linted

I still need to cleanup the README, specifically:

  • the Background Reading section for Mailgun is plain inaccurate :) sorry! This in fact uses Mailgun's official library, not the 3rd party one
  • I want to add a section describing how to use AWS and Mailgun (or any other non-SES mailing service, if added in the future) in the same deployment by specifying the CURRENT_SERVICE env var (basically, the way I set lib/service.js up, it tries to guess the service someone wants to use based on their env vars; in the case of multiple validly configured services, it goes with the last service alphabetically (SES); if, for example, someone was using S3, so would have AWS' standard env vars set, but Mailgun for emailing, the tool would try (and fail) to use S3 instead; CURRENT_SERVICE allows specifying the service you want to use, overriding that default behavior

but I figure I'll hold off on that final documentation cleanup until after handling whatever revisions you want here, once the code's in a place you're happy with

thanks again for letting me take this on! 🙏

@zemccartney
Copy link
Copy Markdown
Author

Ay! My bad on the Travis checks. I probably should've thought about needing to update the .env file there :)

Do I need to handle updating that or do you prefer to? Whichever's good! let me know if I can help with that

@zemccartney zemccartney requested a review from nelsonic February 21, 2018 14:13
@zemccartney
Copy link
Copy Markdown
Author

@nelsonic heyo! do you need anything more from me here? or all set to review?

No rush at all! I know you're busy and this is a side-thing, checking in mostly so I don't forget about this. If the answer's just that I should be patient, I can do that, too :)

Hope you had a great weekend!

@zemccartney
Copy link
Copy Markdown
Author

Heyo @nelsonic ! anything I can do to push this forward?

again, i know it's small potatoes, no big rush, mainly curious :) Thanks again for the help here. Hope you've been having a good weekend 🙏

@zemccartney
Copy link
Copy Markdown
Author

zemccartney commented Dec 25, 2018

@samhstn @nelsonic hey! sorry, I'm super late here, but I finally cleaned up that merge conflict, all tests are back to running, and README is up-to-date.

Travis checks aren't passing, I assume, because some new env vars need to be set for Mailgun. Lemme know if I can help with updating that at all (i.e. if my code / instructions ended up totally unclear).

How is this looking to you?

Copy link
Copy Markdown
Member

@samhstn samhstn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zemccartney Love the updates to the documentation, very clean with loads of little improvements 👍
Let me know your thoughts on the couple of comments I've left for you - I'm happy to approve once answered and travis is fixed
Thanks for your work so far on this ❤️

+ [ ] borrow the code for `hello.txt` and `hello.html` from the **/examples/templates** directory of this project!
+ [ ] create a file called `welcome.js` and paste some sample
code in it (see: [/examples/templates/**send-welcome-email.js**]() )
code in it (see: [/examples/templates/**send-welcome-email.js**](/examples/templates/send-welcome-email.js) )
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice 👍

If you wish to send to multiple recipients of include CC or BCC recipients,
use the sendMany method. This allows you to provide an options object
with an array of `toAddresses`, `ccAddresses`, and `bccAddresses` and charset options.
If you wish to send to multiple recipients or include CC or BCC recipients,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice typo catches 👍

### Which Email Service Provider?

We are *currently* using ***AWS SES*** for ***dwyl***.
We also support **Mailgun**
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

lib/index.js Outdated
var COMPILED_TEMPLATES = {}; // -> cache compiled templates

AWS.config.region = process.env.AWS_REGION;
var send = require('./service').send; // -> email service, for sending
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving aws specific requires and configuration into lib/service.js makes sense 👍

lib/index.js Outdated

ses.sendEmail(params, callback);
// Deep clones options, ensures no mutation of input (deep necessary because source may contain arrays)
var clean_options = JSON.parse(JSON.stringify(options));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zemccartney could you just explain why a deep clone was needed here? As far as I'm aware we weren't mutating the options passed in before. At which point did this become necessary? - I just think it would be great if it was possible to avoid this.

(I quite liked how easy these lines were to read before: https://github.com/dwyl/sendemail/blob/master/lib/index.js#L77-L85)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samhstn hey, thanks calling this out. definitely a bit of a kludge. my bad on that.

I used a deep clone here because I needed to end up with an object containing all the values of the options object that I could then pass as the first parameter to the send call. Since we're running those conditional checks and filters on the properties of that object and need to store those results, I think I thought that meant we'd be mutating the input options (sorry, it's been long enough since I wrote this that I've since forgotten / am a bit lost myself on what I was thinking :) ).

I tried to clean this up a good bit in the work I just pushed up. Still creating a new object, but not cloning our options object. Hopefully readable, little less misdirection here. How does that look?

lib/service.js Outdated
* @returns {Object} interface for parameterizing and sending email send requests
* with the determined service
*/
function determine_service (specific_service) {
Copy link
Copy Markdown
Member

@samhstn samhstn Dec 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a feeling there is maybe a nicer way to do this, having this as part of the library code and referencing another CURRENT_SERVICE env var feels a little off, let me know your thoughts @nelsonic @Danwhy @zemccartney?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samhstn totally agree, I was never all that happy with how I did things here.

I came up with a new approach to this in the stuff I just pushed up.

I still have the library using another env var, now renamed to SENDEMAIL_SERVICE to hopefully make the var's name more descriptive/understandable, but only if the user happens to have multiple services configured at once. In that case, we throw an error saying the user's env config is ambiguous, directing them to use that variable to tell the library which sending service to use.

In short, I tried to clarify and minimize the purpose of that env var by moving it out of the main path of the determine_service path.

I think I was solving for the, to be honest, nonsensical defaulting behavior I had in place to handle this case before (essentially, unless the user had set CURRENT_SERVICE, the library would use the last available service, in alphabetical order....no good 😆)

Does that get to root of this issue? Or at least improve things a bit?

My bad if not! I just jumped to this approach to try working through another thought.
I kept determine_service for now because I figure we need to handle the case — I hope a seriously unlikely one, but I guess still possible — that a user could configure multiple services at once, like our tests. If that's not important, I bet we can simplify this part of the library a good deal (though not sure of the clearest route for that aside from continuing to use an env var...would have to think about that)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samhstn yo! any thoughts here?

sorry, no rush at all. I don't mean to bug, more chiming back so wrapping this up doesn't completely fall off my radar (again :) ). Let me know if there's anything I can do to help move this forward. Thanks for all the help so far

Copy link
Copy Markdown
Member

@samhstn samhstn Jan 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a read through the code, thanks for the improvements here with comments and the explanations - very helpful.

I just want to raise #83 (review) as a concern before any further reviewing or work is done in this pr.

// TODO Delete this test / Or rather, switch to using CURRENT_SERVICE
// TODO Add a test for using CURRENT_SERVICE
// TODO Ok assumption?????
// Assumes .env used for testing has all available services configured simultaneously
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zemccartney what is your plan with these TODOs, are you planning to address these in this pr or should we open an issue to work on these in a later pr?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heh...sorry for that sloppiness. those are all resolved, I did a pass over the service testing file. I added a handful of new stuff, but all things that make the tests more reliable and readable.

@samhstn
Copy link
Copy Markdown
Member

samhstn commented Dec 26, 2018

In regards to the environment vars, we currently just have these in travis:

AWS_ACCESS_KEY_ID 
AWS_REGION
AWS_SECRET_ACCESS_KEY
SENDER_EMAIL_ADDRESS
TEMPLATE_DIRECTORY

Looks like we just need to add these two for the tests to pass:

MAILGUN_API_KEY
MAILGUN_SENDING_DOMAIN

@zemccartney
Copy link
Copy Markdown
Author

zemccartney commented Dec 26, 2018

Hey @samhstn ! Very cool, thanks for all those detailed comments. I appreciate you taking the time to review this work and write those up 🙏

I'm going to go back and leave responses, but I just pushed up another pass that I think addresses all the issues you called out.

Basically,

  • gets rid of that deep clone, made that options validating / massaging in sendMany more closely resemble how it was before, hopefully a bit cleaner.
  • refactors how we handle multiple services registered simultaneously. I moved from essentially randomly picking one to throwing an error that directs the user to set an env var to declare which service they want to use. More on this
  • cleans up the tests a bit, closes those open TODOs

FINALLY (sorry to drag on here :) ), to your comment about setting new env vars, that's correct, but you'll also need to add MAILGUN_AUTHORIZED_RECIPIENT
This is a bit kludgy, I admit, but is used in testing only. Because Mailgun doesn't have a success simulator equivalent that I know of and to send to any inbox while in sandbox mode you need to invite that inbox to be an authorized recipient from your Mailgun dashboard, I've been authorizing a personal email as a recipient, then setting the authorized recipient var to that inbox. Does that work ok for you all?

@shaneparsons
Copy link
Copy Markdown

Is this ready to be merged? I'd love to use this in a current project!

@nelsonic
Copy link
Copy Markdown
Member

nelsonic commented Jan 4, 2019

@zemccartney code is looking good. 🎉
@samhstn thanks for reviewing with a fine-tooth-comb. ❤️
@shaneparsons we need to ensure that we have the necessary environment variables defined on TravisCI to ensure that this build will always pass on CI ... we need to add an "admin task" for this. ⏳

@zemccartney
Copy link
Copy Markdown
Author

@nelsonic awesome, glad to hear! thanks so much for the feedback 🙏 let me know if there's anything at all I can do to help with the CI task

Copy link
Copy Markdown
Member

@samhstn samhstn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zemccartney thanks for the code changes here, finding it much nicer to read through.

These issues are a little concerning though: mailgun/mailgun.js#43 and mailgun/mailgun.js#39. It seems that the mailgun.js npm module hasn't been updated in 2 years and has security vulnerabilities.

Although we are using a small subset of what the mailgun api has to offer, we should at least investigate what the current security vulnerabilities are before merging this or perhaps consider a different library, one which is maintained.

@zemccartney
Copy link
Copy Markdown
Author

👍 awesome, thanks for pointing that out @samhstn sorry for my oversight there.

IIRC, way back when I first started this, I debated between using that library and this community-contributed one: https://github.com/bojand/mailgun-js

I think I switched to Mailgun's official one, assuming it'd be more reliable, but looks like even Mailgun's official node tutorial directs people to use the community library :) Doi. On reviewing that library, it looks like it's still maintained (at least relatively recent issues and commits).

So I'm thinking I'll switch out Mailgun's client for the community one, make whatever updates are necessary to get that working. As part of that work, I'll audit that project's dependencies, see if there are any known security issues there, contacting the maintainer about my findings as needed.

How does that sound to you?

@samhstn
Copy link
Copy Markdown
Member

samhstn commented Jan 5, 2019

Sounds good @zemccartney, yeah the community contributed npm module looks better to use 👍thanks for looking into this ❤️

@zemccartney
Copy link
Copy Markdown
Author

@samhstn heyo! sorry for the long delay on this, hit a busy stretch. The commit I just pushed up replaces the unmaintained mailgun client with the maintained one and updates dependencies based on the results of npm audit (just tap-spec needed bumping).

Additionally, I noticed how I'd written things previously was a bit inefficient — I had been determining a service and instantiating a sending client on every send — so I refactored the main export of the service module (the service-specific sending function) to enclose just a call to the determined service's sending method. The upside is that this is a bit more efficient, I imagine. Downside is that you can no longer switch your sending service mid-process without clearing sendemail from require's cache and re-requiring it (which is what we do now in the main test file). Not strictly necessary at all! Happy to switch back to how things were, just something I felt was an improvement.

Any questions or anything I can do to help here, lemme know!

Thanks again for all the help so far 🙏

@samhstn samhstn assigned samhstn and unassigned zemccartney Feb 11, 2019
@samhstn samhstn assigned zemccartney and unassigned samhstn Feb 12, 2019
@zemccartney
Copy link
Copy Markdown
Author

Wicked, thanks for the feedback @samhstn 🙏 I'll address those notes soon

@zemccartney
Copy link
Copy Markdown
Author

@samhstn heyo! thanks again for all the feedback. I think I covered everything in the changes I just pushed up, but as always, let me know whatever else more is needed 🙏

@samhstn
Copy link
Copy Markdown
Member

samhstn commented Feb 19, 2019

@zemccartney looks great ❤️ thanks for your work

@nelsonic the build is failing because mailgun-js uses the class syntax which is not supported on node v4, (see: https://stackoverflow.com/questions/41960142/block-scoped-declarations-not-yet-supported-outside-strict-mode)

What do you think we should do about this? Upgrade version or look for another solution?

@samhstn samhstn assigned nelsonic and unassigned zemccartney Feb 19, 2019
### Checklist (*everything you need to get started in 5 minutes*)
+ [ ] install the `sendemail` module from NPM
+ [ ] Ensure that you have an an account on your email service of choice
+ [ ] Ensure that you have an account for your email service of choice
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@zemccartney
Copy link
Copy Markdown
Author

@samhstn @nelsonic my two cents here, whatever it's worth :) this could be a good opportunity to align sendemail's node support more closely with node's current maintenance schedule i.e. since node 4 is no longer maintained, you could bump the lowest version of node supported here to 6 or 8 (both in maintenance LTS for parts of this year ; and it looks like node 6 is the earliest version to allow ES6 class syntax without a runtime flag), and release that update as sendemail v4. Users on older versions of node could still use v3 or earlier.

Of course, I'm partial to this route in part because it means I don't have to do anything :trollface: but I do think trying to keep step with node's maintenance schedule and the patches provided therein is a worthwhile, if exhausting, effort in the interest of both access to new features and security patches.

Sorry if I'm overstepping here! I'll happily butt right back out if so

@nelsonic
Copy link
Copy Markdown
Member

@zemccartney we agree. LTS is the preferred support target.
Our original reasoning for having explicit Node v4 support was AWS Lambda.
We used sendemail on v4 on Lambda.
The current version of Node.js on AWS Lambda is 8.10
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
image

I think we can safely discontinue support for node.js v.4 and even v.6 on the basis that v8.10 is available.

@zemccartney
Copy link
Copy Markdown
Author

@samhstn @nelsonic heyo! any word on moving this forward? not being impatient, no rush from me, just I realized I'd spaced on this, figured worth doubling back to see if there were anything I could do to help wrap this up. Thanks!

@samhstn
Copy link
Copy Markdown
Member

samhstn commented Mar 20, 2019

@nelsonic for our tests to pass, it looks like we need to set these two environment variables in .travis.yml:

MAILGUN_API_KEY
MAILGUN_SENDING_DOMAIN

Do you want me to go ahead and try and add them (I think I have permissions issues)?

@zemccartney
Copy link
Copy Markdown
Author

zemccartney commented Mar 20, 2019

@samhstn yo! thanks for jumping right on this.

re: the new env vars, you'll also need to add MAILGUN_AUTHORIZED_RECIPIENT (see https://github.com/dwyl/sendemail/pull/83/files#diff-0fd0e07cf6d02bf7cf00f18cebb8e6eaR97)
This is a bit kludgy, I admit, but is used in testing only. Because Mailgun doesn't have a success simulator equivalent that I know of and to send to any inbox while in sandbox mode you need to invite that inbox to be an authorized recipient from your Mailgun dashboard, I've been authorizing a personal email as a recipient, then setting the authorized recipient var to that inbox. Does that work ok for you all?

@zemccartney
Copy link
Copy Markdown
Author

@nelsonic @samhstn heyo! sorry, been a while. just following up on this. is there anything I can do to help move this forward? as always, not to pressure at all. More to keep myself from forgetting :) Thanks!

@zemccartney
Copy link
Copy Markdown
Author

@nelsonic @samhstn hey y'all, checking back in on this. anything I can do here to move this forward?

I hope y'all've been well!

Copy link
Copy Markdown
Member

@samhstn samhstn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, think the environment variables need sorting out

@zemccartney
Copy link
Copy Markdown
Author

Awesome, glad to hear, thanks for reviewing!
Anything I can help with on the env var front?

language: node_js
node_js:
- 4.3.2
- 8.1
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn’t it be >12?

Copy link
Copy Markdown
Member

@nelsonic nelsonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zemccartney. 👌
Apologies that this one fell off the radar. 🤷‍♂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants