Limit RESOLUTION_MAX to INT_MAX, not UINT_MAX (all spelled out) so the

mode value isn't always clipped to -1 when (resolution * size) == 32, which
would have been the case with values => {4i,32b,32t}.

This seems to have been broken in r64382.

MFC after: 1 week
X-MFC with: r289915
PR: 200619
Reported by: Michael Baptist
Submitted by: Lars Skodje
Sponsored by: EMC / Isilon Storage Division

sbin/camcontrol/modeedit.c

@@ -246,7 +246,7 @@ editentry_set(char *name, char *newvalue, int editonly)
  *     currently workaround it (even for int64's), so we have to kludge it.
  */
 #define	RESOLUTION_MAX(size) ((resolution * (size) == 32)? 		\
-	(int)0xffffffff: (1 << (resolution * (size))) - 1)
+	INT_MAX: (1 << (resolution * (size))) - 1)
 
 	assert(newvalue != NULL);
 	if (*newvalue == '\0')