You can go back and forth on trying to get a "seamless" paint/varnish edge and never really do so.  And unless the light is just right, realize that this won't be visible/noticable from more than a few feet away.  My suggestion/best success in minimizing the line:

1. Ever so slightly feather down the edge of the "first" side of your finish, whether that be paint or varnish with some fine sandpaper, trying to stay less than 1/8 inch from the edge. I'd suggest it be the varnish first, if possible, because any varnish on top of paint tends to turn yellow and be noticeable.

2. Put on your several coats of paint, using your tape edge laid right along the varnish edge.  Know that you'll be putting one more coat of paint on after this step, so judge coats accordingly.

3. Pull the tape after this next-to-last coat when the paint is still wet, and after it is dry every so slightly feather down the paint edge.

4. Using your good fine line tape, tape up over your first laid finish (the varnish?) edge by 1/8 inch. Lay down your final coat of paint. This will mean you are now expanding the area to be painted by just a bit, and that this small addition will only be receiving one coat.

5. Carefully pull the tape while the last coat of paint is still wet. Pull the tape just as soon as you can get to it after getting the paint on. You will get some natural flow and softening of the edge. 

6. After the paint dries but before it is hard you might try going along the edge with your thumbnail or a popsicle stick, if necessary, to see if that helps press down any remaining edge.

7. Wait at least 2-3 weeks and add a coat of wax.  This is controversial - some like waxing, some don't - I like waxing. If things work out well, by the time wax fills any of the slightest parts of the mated finish edge, you won't even be able to feel/notice it.

And back to the top, this is probably all overkill. I mostly just butt-joint the varnish and paint edges (no fair-sainding), be sure to pull the tape on the final coat of paint while still wet, run a thumbnail over it while soft, later wax, then go boating.

   Thanks for the informative reply. Never varnished or painted anything other than a house. I live in a wooden boat vacuum so this really will help me finish this beauty with some level of confidence. I prefer leaning from other people's experience than trying to figure it out by trial and error. SB

   I agree that you are veturing into overkill, or at the least, over work territory with this concern. After the varnish is througoughly dry, tape the varnish side along the - to be painted edge - as carefully, accurately and tightly as you can. When you add your 2 or 3 coats of paint to that tape line you will be creating a relatively equal height edge, i.e., an edge that will for the most part be as “tall” as the distinct varnish edge you are concerened about. To my way of seeing it - and I am particular - these two mating edges will blend their heights together imperceptibly. My 2¢. Have fun with it.