about
articles
projects
blogroll
bit-101
coding horror
blameit
failblog
beszeljukmac
sghu
posts
docs
admin
|
2010-07-30 Synthesizing sound on flash, java, macos and ios platforms (iphone,ipad,ipod) |
|
Sound waves are continuous pressure changes in the air. In digital technology, we call these changes "pulses", and change count per second is called "sampling rate".
The standard raw format for digital audio is "pulse code modulating" format (PCM), which is simply a queue of amplitude values which tells the hardware how powerful signals should it produce for output. The human brain can process more than 100.000 changes per second. However, because of historical reasons the most common sampling rate in digital technology is 44100 samples per second. So most of the time coders have to deal with this sampling rate, on every channel, and the usual bit depth is 16. This means that one pulse in the pulse stream can have 2^16 levels. So one channel ( a mono stream ) uses 2 bytes per frame, and a stereo stream, which contains two channels, takes 4 bytes per frame. That's all we have to know to start experimenting. We are going to make a simple sound generator, which generates a simple musical "A" tone. The musical "A" tone is a simple sinus wave, having 440 periods per second. I separate the logic in two classes : the WaveGenerator, and the WavePlayer. The WaveGenerator does nothing spectacular : it will generate a sinus wave on the given frequency sampled at the given sampling rate. The platform dependent magic will be in the WavePlayer class, it will contain the hardware initializer code, and the platform-specific instructions. The scheme for WaveGenerator class is this : field position // actual position/angle filed previous // previous position/angle field increment // position increment per step field frequency // tone frequency method construct ( samplingrate, tonefrequency ) { increment = 2 * PI / samplingrate; // calculate increment frequency = tonefrequency; } method nextAmplitude ( ) { if ( position > 2 * PI ) position -= 2 * PI; // rolling back position/angle return Math.sin( previous * frequency ); // passing back sinus of actual position multiplied by frequency } That's all, it's quite straightforward, let's check it out on all platforms : flash WaveGenerator.as java WaveGenerator.java mac/ios WaveGenerator.m Let's see the WavePlayer class. We are heading from the easiest platform to the hardest. Flash In flash you don't/can't do any low-level setup. It supports only 16 bit depth stereo sounds sampled in 44100 Hz, but you have to provide only floating numbers between -1 and 1 to the API for pulses. The worklfow is the following : Create a new instance of the Sound class. Add an event listener to this class, which listens for SAMPLE_DATA events. These events will fire when the hardware runs out of buffer, and it needs more data. In the event handling function add more data to the buffer. Finally, you call soundInstance.play( ). Event firing will never be accurate, so we mustn't hang on timing, we have to use a separate frame counter, or we can use the event's position field, which is the same. If we push 4410 frames to the buffer, the next event will come around 4410/44100 = .1 seconds. As i said before, we fill up the buffer using float values from -1 to 1, and our WaveGenerator class does exactly this. So let's see the code : WavePlayer.as Java In java we have to dive to a lower level. We will use the Java Media Framework. In java we have to ask for specific data lines ( input streams ) to specific audio mixers, and have to push data to them continuousliy, like in flash. To get a specific data line, we have to specify an audio format first, and AudioSystem class will give us a data line if our hardware is capable of playing sound in the provided format, or nothing, if not. We use the most common format mentioned before : 16 bits, 44100 Hz sampling rate, 2 channels, 4 bytes per frame. In java, we have the possibility to set the integer type for the samples, byte ordering, and playing frame rate, which should be the same as the sampling rate. AudioFormat format = new AudioFormat( AudioFormat.Encoding.PCM_SIGNED , 44100 , 16 , 2 , 4 , 44100 , true ); Then we can start playing the dataline, and providing frame data for the input stream. To make playing more solid, i've created a fillBufffer function, which pre-creates buffer parts for every .1 seconds. We have to calculate buffer size and frame count before this. bufferSize = (int)(44100 * 4 * .1 ); // .1 second length buffer frameCount = bufferSize / 4; fillBuffer function will convert float values returned from WaveGenerator to signed 16-bit ( short ) values using Short.MAX_VALUE : short sample = (short) ( source.nextAmplitude() * Short.MAX_VALUE ); and pushes them into the bytearrayoutputstream in stereo : stream.write( sample >> 8 ); stream.write( sample ); stream.write( sample >> 8 ); stream.write( sample ); And that's all. Check it out : WavePlayer.java Mac/iOs And now for something completely different, and difficult. On mac/ios you have to use the CoreAudio framework. It is not inputstream-based like the flash/java solution, here you have to provide and arbitrary number of buffers for sound, and CoreAudio will rotate them, and notifies you in a standard C call if a buffer is emptied, and you have to fill up that buffer while it plays the other buffers, so it's really cpu friendly and effective, but quite difficult to handle. First we have to define the audio format. It's a simple structure, with additional options compared to java, let it speak for itself : audioFormat = (AudioStreamBasicDescription) { .mFormatID = kAudioFormatLinearPCM, .mFormatFlags = 0 | kAudioFormatFlagIsPacked | kAudioFormatFlagIsSignedInteger | kAudioFormatFlagsNativeEndian , .mSampleRate = 44100, .mBitsPerChannel = 16, .mChannelsPerFrame = 2, .mBytesPerFrame = 4, .mFramesPerPacket = 1, .mBytesPerPacket = 4, }; bufferSize and frameCount are calculated in the same way bufferSize = audioFormat.mSampleRate * audioFormat.mBytesPerPacket * .1; frameCount = bufferSize / audioFormat.mBytesPerFrame; this is followed by audio session initialization, and setting up the audio queue output : AudioQueueNewOutput( &audioFormat, // audio format updateQueue, // callback function on buffer update request self, // custom parameter, passing ourself NULL, // timing comes from the audioqueue's thread NULL, // deafult loop mode 0, // flags &audioQueue // audio queue which stores the created session ); updateQueue has to be a plain C function, and since we started from an objective-c class, we have to place it over the @implementation part. It contains nothing but a simple callback to the objective c function - ( void ) updateQueue : ( AudioQueueRef ) pAudioQueue updateBuffer : ( AudioQueueBufferRef ) pLastBuffer where we can handle buffer fillup and rotation. After setting up the queue, we fill up the starting buffers, and call AudioQueueStart(audioQueue, NULL); and the process starts. Let's check out the updateQueue:updateBuffer: method. Like in java, we convert the floating point values to signed 16-bit integers. short sampleValue = [ waveSource nextAmplitude ] * INT16_MAX; And at the end, we push the buffer back to the queue. AudioQueueEnqueueBuffer( pAudioQueue , pLastBuffer , 0 , NULL ); Check out the whole class : WavePlayer.m And that's all folks! Feel free to download the specific projects : flash Sound Synthesizer ( Flash Builder 4 ) java Sound Synthesizer ( Eclipse ) mac/ios Sound Synthesizer ( xcode ) |
|
2010-01-18 iPhone openGL ES basics - animation |
|
Lets continue from "iPhone openGL ES basics - textures as point sprites"
Lets move our point sprites around. We need a few instance variables for this in the header. We need an array of GLfloats to store coordinates, two float for storing the two particles directions, the EAGLContext, and a NSTimer for simulation/screen refresh. #import "GLView.h" #import <OpenGLES/ES1/glext.h> @interface MinimalAppDelegate : NSObject { NSTimer* timer; UIWindow* window; GLView* glview; GLfloat* points; EAGLContext* context; float angleA; float angleB; } - ( void ) update; @end Go back to MinimalAppDelegate implementation. We have to remove the drawing part from the bottom of applicationDidFinishLaunching method. We have to setup movement variables and the timer there instead. angleA = ( float ) rand( ) / RAND_MAX * M_PI * 2; angleB = ( float ) rand( ) / RAND_MAX * M_PI * 2; points = malloc( sizeof( GLfloat ) * 4 ); points[ 0 ] = ( GLfloat ) rand( ) / RAND_MAX * 320; points[ 1 ] = ( GLfloat ) rand( ) / RAND_MAX * 480; points[ 2 ] = ( GLfloat ) rand( ) / RAND_MAX * 320; points[ 3 ] = ( GLfloat ) rand( ) / RAND_MAX * 480; // START TIMER timer = [ NSTimer scheduledTimerWithTimeInterval : ( NSTimeInterval ) 1.0 / 20.0 target : self selector : @selector ( update ) userInfo : nil repeats : YES ]; We have to move the drawing part to a separate function, which will be called by our NSTimer simultaneously. This will be "update". First we need to refresh particle positions. // modify direction angleA += -.5 + ( float ) rand( ) / RAND_MAX * 1; angleB += -.5 + ( float ) rand( ) / RAND_MAX * 1; // update position points[ 0 ] += sin( angleA ) * 10; points[ 1 ] += cos( angleA ) * 10; points[ 2 ] += sin( angleB ) * 10; points[ 3 ] += cos( angleB ) * 10; // check borders if ( points[0] > 320 || points[0] < 0 || points[1]> 480 || points[1] < 0 ) angleA -= M_PI; if ( points[2] > 320 || points[2] < 0 || points[3]> 480 || points[3] < 0 ) angleB -= M_PI; Then let's move the texture sprite drawing code below // enable texturing glEnable( GL_TEXTURE_2D ); // assig vertex pointer glVertexPointer( 2 , GL_FLOAT , 0 , points ); // set point size glPointSize( 150 ); // draw vertexes glDrawArrays( GL_POINTS , 0 , 2 ); Then, to make things more spectacular, we darken the whole scene, and we get a nice "glowing trail" effect. We will draw a simple square with the help of a triangle strip, we will use vertex coloring, so we have to switch off textures, and enable color array access. The last part is : // disable texturing glDisable( GL_TEXTURE_2D ); // enable color array for writing glEnableClientState( GL_COLOR_ARRAY ); // define screen sized square with transparency GLfloat square [ ] = { 0 , 0 , 0 , 480 , 320 , 0 , 320 , 480 }; GLubyte colors [ ] = { 0 , 0 , 0 , 200 , 0 , 0 , 0 , 200 , 0 , 0 , 0 , 200 , 0 , 0 , 0 , 200 }; // assign vertex and color pointer glVertexPointer( 2 , GL_FLOAT , 0 , square ); glColorPointer( 4 , GL_UNSIGNED_BYTE , 0 , colors ); // draw glDrawArrays( GL_TRIANGLE_STRIP , 0 , 4 ); // disable color array before using texture glDisableClientState( GL_COLOR_ARRAY ); // display renderbuffer content [ context presentRenderbuffer : GL_RENDERBUFFER_OES ]; If you move the darkening part before the sprite drawing, you get brighter glows, check it in the source. Download source. |
|
2010-01-17 iPhone openGL ES basics - textures as point sprites |
|
Okay, let's continue from "The minimal iphone opengl es application".
Let's go to the point in MinimalAppDelegate where we bind glview's layer to opengl's renderbuffer. [ context renderbufferStorage : GL_RENDERBUFFER_OES fromDrawable : ( CAEAGLLayer * ) glview.layer ]; So we want some texture. The simplest form of a texture in opengl is a point sprite. OpenGL maps the given texture to the wanted point with the wanted size. Sounds simple, and it is. First we have to load an image we want to use as a texture. Create a gradient sphere with alpha in a photo editor, save it as a transparent png. We use UIImage for loading the image. CGImageRef brushImage = [ UIImage imageNamed : @"Particle.png" ].CGImage; We have to extract the raw byte data from the image, we use Core Foundation's DataGetBytePointer function to get the pointer to the byte array provided by CGImage's dataprovider. GLubyte* brushData = ( GLubyte * ) CFDataGetBytePtr( CGDataProviderCopyData ( CGImageGetDataProvider ( brushImage ) ) ); We have it now, texture generation comes GLuint brushTexture; glGenTextures( 1 , &brushTexture ); glBindTexture( GL_TEXTURE_2D , brushTexture ); Let's set up a linear filter for the texture glTexParameteri( GL_TEXTURE_2D , GL_TEXTURE_MIN_FILTER , GL_LINEAR ); And finally create the texture from our bytearray glTexImage2D( GL_TEXTURE_2D , 0 , GL_RGBA , CGImageGetWidth( brushImage ) , CGImageGetHeight( brushImage ) , 0 , GL_RGBA , GL_UNSIGNED_BYTE , brushData ); Yaaay. Don't forget to free brushData array. We have the texture, let's draw it as point sprites.We want to see smooth alpha transitions, so we have to enable alpha blending. glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE); Let's set up the screen's dimensions for orthographic projection and for the viewport. glMatrixMode( GL_PROJECTION ); glOrthof( 0 , 320 , 0 , 480 , -1 , 1 ); glViewport( 0 , 0 , 320 , 480 ); Enable 2D textures and point sprites. glEnable( GL_TEXTURE_2D ); glEnable( GL_POINT_SPRITE_OES ); glTexEnvf(GL_POINT_SPRITE_OES, GL_COORD_REPLACE_OES, GL_TRUE ); Set up points for the sprites glPointSize( 100 ); GLfloat points [ ] = { 130 , 200 , 100 , 204 }; Enable vertex array for writing glEnableClientState(GL_VERTEX_ARRAY); Assign vertex pointer glVertexPointer( 2 , GL_FLOAT , 0 , points ); And finally, draw vertexes. glDrawArrays( GL_POINTS , 0 , 2 ); And display renderbuffer. [ context presentRenderbuffer : GL_RENDERBUFFER_OES ]; Run/debug it, you'll see two beautiful intersecting clouds :D Download source. MilGra |
|
2010-01-16 The minimal iPhone openGL ES application |
|
Let's use the source from the previous tutorial, "The minimal iPhone application".
We will need an openGL ES framework. Right click on Frameworks folder, Add -> Existing Frameworks, select an OpenGLES framework, click Add. If you don't know how to locate one, create a new openGLES project, right click on OpenGLES.framework under frameworks, and check the full path. We also need the QuartzCore framework for CAEAGLLayer. Do as above. To show an openGL output, we need a core animation gl layer to bind the renderbuffer to, and UIView has this. So we need an UIView instance. But its not that simple, by default UIView's layer is a CALayer instance, but openGL context needs a CAEAGLLayer instance to work, so we have to mimic it. We need a new class which extends UIView, and we have to override UIView's "layerClass" class method definition. Right click on Classes, Add -> New File -> Objective-C class -> Next -> name it GLView.m, click Finish. Delete the #import from the header file. Extend it from UIView. #import @interface GLView : UIView { } @end Only one thing is needed in the implementation file, the layerClass override : #import "GLView.h" @implementation GLView + ( Class ) layerClass { return [ CAEAGLLayer class ]; } @end So from now on it will tell that its layer is a caeagllayer instance. Great. We made two additional files for almost nothing. Quite a shitty solution from Apple. Yesokay, lets get back to MinimalAppDelegate. We have to instantiate our layer. We need a new property in the header : GLView* glview; don't forget to import #import "GLView.h" we also need to import the openGL ES1 extensions #import <OpenGLES/ES1/glext.h> The result : #import "GLView.h" #import <OpenGLES/ES1/glext.h> @interface MinimalAppDelegate : NSObject { UIWindow* window; GLView* glview; } @end And in the implementation we instantiate our newly created GLView class: glview = [ [ GLView alloc ] initWithFrame : [ [ UIScreen mainScreen ] bounds ] ]; Lets add this view as a subview to the main window. [ window addSubview : glview ]; now we can create our openGL ES1 context. EAGLContext* context = [ [ EAGLContext alloc ] initWithAPI : kEAGLRenderingAPIOpenGLES1 ]; Set it as the active context : [ EAGLContext setCurrentContext : context ]; From now on we can work with the context. We need a color and a frame buffer. Let's generate names for them. GLuint colorBuffer; GLuint frameBuffer; glGenFramebuffersOES( 1 , &frameBuffer ); glGenRenderbuffersOES( 1 , &colorBuffer ); Bind framebuffer as framebuffer, colorbuffer as renderbuffer. glBindFramebufferOES( GL_FRAMEBUFFER_OES , frameBuffer ); glBindRenderbufferOES( GL_RENDERBUFFER_OES , colorBuffer ); We have to attach the colorBuffer to the frameBuffer as color attachment. glFramebufferRenderbufferOES( GL_FRAMEBUFFER_OES , GL_COLOR_ATTACHMENT0_OES , GL_RENDERBUFFER_OES , colorBuffer ); The only thing left is binding glview's layer to opengl's renderbuffer. [ context renderbufferStorage : GL_RENDERBUFFER_OES fromDrawable : ( CAEAGLLayer * ) glview.layer ]; Now we can do something spectacular, for example, setting the background color to dark green. glClearColor( 0 , .3 , 0 , 1.0 ); glClear( GL_COLOR_BUFFER_BIT ); And display the render buffer : [ context presentRenderbuffer : GL_RENDERBUFFER_OES ]; Yaaaay! If you run/debug the application, you will see a beautiful dark green color on your opengl layer. Download source. MilGra |
|
2010-01-15 The minimal iPhone application |
|
"Keep it as small and simple as possible" - me, all the time
Xcode, New Project -> Application -> Window Based Application, name it Minimal, and Save. What you see in xcode now is the plain iphone application. But there are still too many unnecessary things there. We are real programmers, we won't need the interface builder descriptor, so right click on MainWindow.xib, select delete, and move to trash. Open Minimal-Info.plist as plain text file, and delete the last key-string node from it ( with MSMainNibFile and MainWindow ). Let's remove Coregraphics.framework from frameworks, we don't need it. Right click on it, delete, move to trash. Let's check main.m. Delete NSAutoreleasePool initialization and the release lines, we are real programmers, we want total control, not autorelease sh*t. :) Without these two lines we can simplify this function, move UIApplicationMain init after return, and remove UIKit import from the top, its already in the prefix header. We also have to tell UIApplicationMain which is our Delegate class. The result should look like this : int main ( int countX , char *wordsX[ ] ) { return UIApplicationMain( countX , wordsX , nil , @"MinimalAppDelegate" ); } Let's check MinimalAppDelegate.h. Delete the @property line, we don't want to set, get and synthesize the window. Also remove UIKit import. MinimalAppDelegate.m comes. Remove @synthesize window. Let's create it "manually". Normally it is synthesized based on the interface builder file, but we deleted that. Let's create an UIWindow instance with the screen's dimensions : window = [ [ UIWindow alloc ] initWithFrame : [ [ UIScreen mainScreen ] bounds ] ]; The result : @implementation MinimalAppDelegate - ( void ) applicationDidFinishLaunching : ( UIApplication* ) application { window = [ [ UIWindow alloc ] initWithFrame : [ [ UIScreen mainScreen ] bounds ] ]; [ window makeKeyAndVisible ]; } - ( void ) dealloc { [ window release ]; [ super dealloc ]; } @end You can debug and run the application. Let's check the project's folder. We have three files in the root, the prefix, the plist and main.m, and two files for MinimalAppDelegate class under classes folder. If you do a release build, you will find that the size of the binary is 17704 bytes, so this seems the minimal size of a compiled application, of course you can short it with one-char method and attribute names, but it won't be readable then, so it won't be that simple to work with, and our first and only rule would be harmed. Here is the link to the project source. MilGra |
|
2009-11-01 Termite is out!!! |
|
After three weeks of heavy development and objective-c exploration my first iPhone game is out :D
Termite It wasn't easy. After ten years of java/actionscript i had to rediscover c, and learn its smalltalk like objective extensions. Then a lot of experimenting came, and i started writing Termite under the project name "The Swarm". In the first phase i used Quartz for rendering, but after the first few crowded scenes i had to consider using openGL, because Quartz was really slow over 2-300 particles, and openGL can easily render 10000 on a 3G. Of course, 10000 ants wouldn't be possible, beacuse of the continuous proximity detection. I had a few issues with that, first every actor cheked every actor, and it became really slow because of the tremendous array enumerations, so i turned to collosion grid, and the speed became fair. After finishing the engine, i realized that i will need fix levels, not just random and custom ones, so in the last two weeks i was figuring out new levels. In the last week I bought a worn down scratchy used iphone, the cheapest in the market, and started testing, and everything was fine. So, I did the final steps, compiled the app with the distribution certificate, and its out. I know the gfx is quite modest, but its old school man!!! :D I'm sure the playability is okay, i really love playing this on the subway. So keep it playing while my next game ( which will be 3D, yeaaaah ) is coming. MilGra |
|
2009-06-22 One of the best programming tools ever made!!! |
|
And that is sdedit. The quick sequence diagram editor. Its awesome, its heaven, its salvation.
Its like a command line uml sequence diagram drawer. There is a descriptor script on the left, and the diagram on the right. Only with a few lines of description you can create complex diagrams, it supports multithreading, and everything possible. I was in pain in the past four days, i’ve tried a lot of sequence diagram editors, standalone and eclipse-plugins, free and closed source, but they were all so difficult to use, click a lot for a little change, to insert a thread, or it was impossible at all. And then i found sdedit. I was surprised, is it possible to describe complex diagrams with simple commands? And yes, since it draws sequence diagrams, sequencing elements is straightforward. Should be industry-standard. Thank you Marcus Strauch. 
|
|
2009-06-10 Quick flash to java switch |
|
I would like to help out those few sad souls who want to switch from flash/flex to java to create some graphical stuff.
I just want to show the basic idea, let’s draw a simple red square. In flash its simple, since flash is for creating graphics, the main class has to be a Sprite or a MovieClip, and you can directly access the graphics interface of these two, and can draw directly. In flash, since the first frame is the main sprite, you simply can write graphics.beginFill( 0xff0000 , 1 ); graphics.drawRect( 0 , 0 , 100 , 100 ); in flex, it looks like this : Source Code and there is it. In java, since its a general purpose programming language, its more complicated. We need a window, where we can show our graphics, and then we need a component where we can paint our stuff. But still, we can do it quite simply in one class. To create a window, we simply instantiate java.awt.Frame class, and for drawing we will use the java.awt.Canvas class, which is the simplest component for drawing. Consider java.awt.Frame as the Stage in flash, and java.awt.Canvas as the main Sprite. So, here we go : Source Code The biggest difference that we cannot draw immediately, we have to wait for jvm’s inner call to the paint( Graphics g ) method, altough we can request it by calling the repaint( ) method. We have an easier job with an applet, because Applet main class is a Panel subclass in which we have the paint method, and its already in a window. MilGra |