To be completely honest, sometimes I wonder why we include limits in the curriculum at all, at least at the place we do.
The traditional place they are found is just before derivatives, which makes mathematical sense, since the definition of derivative is a limit. However, almost all of the students I talk to individually (several hundred a year), keep the ideas of limits and derivatives completely separate in their minds, at best treating limits as an annoying thing they did before they got to the real business of differentiating.
I would much prefer that differentiation was taught as a process for finding a formula for a rate of change of a function at particular points -- a new function where when you sub in the point, it tells you the rate of change. Through much observation our forefathers realised there were all these neat relationships for various functions and combinations thereof, and also that it could be used to solve all sorts of useful problems. In essence you teach differentiation straight up as something that you do to functions. I have no evidence to suggest this is the best approach, but I have much empirical evidence that this is how students already think about it.
I'm not necessarily suggesting to completely remove limits and the definition of derivative from the curriculum (though it probably wouldn't hurt your non-maths majors too much to do so). What I am suggesting is to get them familiar with the mechanics of how they behave and the usefulness of them before getting into the foundations of mathematical rigour it's built on.
I can imagine, after you've gotten them smoothly doing derivatives, that you could have a section of the course where the explicit aim is to lay the mathematical foundation for derivatives and other concepts of function. This is where limits belong -- they belong to the general discussion of continuity and differentiability. From the raw meaning of function, what does it mean to say a function is continuous? What does it mean to say a function has a derivative? What sorts of things can you generally say about functions and their derivatives and when and where they exist? (That is, theorems like the Mean Value Theorem.) These concepts only really make sense to ask about when you have already got a good understanding of the behaviour of functions, and indeed they will probably make more sense because you can use their already-existing intuition of what ought to happen to support their learning of what does happen.
If the purpose was to actually discuss the mathematical foundations -- after having built something that is worthy of having a foundation -- then I don't see a problem using epsilons and deltas. It would seem fitting if we were going to be rigorous to really do it properly. You could choose how quickly you switched from that to the limit laws based on the level of your students, of course.
It would of course be important that any time you did ask them to prove anything, that you were clear on what they were and were not allowed to use -- for example, the epsilon-delta definition or the limit laws; differentiating using the rules or with limits?