solve the magnitude of two force such that if any act at right angle resultant is 5kn while the act at angle60 their resultant is square 37

Let the two forces be

ax and by

a^2+b^2 = 25
a^2 + b^2 - 2abcos120° = 37
or
a^2+b^2+ab = 37

ab=12

So, it looks like the two forces are 3kN and 4kN