A square for males, circle for females, is the standard for genogram family diagrams, used in medicine, genetics and social work.
However, these diagrams are rarely used in genealogy (so far). Genopro is one of the few programs that does generate them.
Although genograms can use colours on lines to indicate types of relationships, they are also slightly different line patterns. If you used more features of genograms, beyond the simple gender indicator shape, they would work best with colour but you would still be able to use black and white copies.
Here's a short lesson from FamilySearch on Tracing Family Traits Using a Genogram (video and pdf) that has the basics how to create genograms, and more references. With genetics increasingly becoming a part of genealogy, genograms may become more used in the future.
Genograms normally have the names outside of and below the square/circle rather than inside, see some examples. That may be too big a change to the look of your chart though.
The rounded corners convention is the other option if you need boxes to put content in (see an example). I don't think it's ever been described as a formal standard, but it's not unknown.
The Mars/Venus symbols were used in medicine in mostly Europe in the early 20th century but I haven't seen them used in modern genealogy charts, they're not a standard or convention. It may be difficult to see the difference between them at small sizes (which is also possibly a problem with the rounded corner rectangles). See a discussion of historical pedigree typography (pdf).
In summary, the squares and circles of geneotypes are an international standard but it's not widely used in genealogy (possibly because it can look quite ugly with the text outside). The curved box corners for a female does exist as a convention, and is easy to do. Using Mars and Venus symbols in the background isn't a standard or convention and might make the text on them hard to read.