中财网保姆毒死老人审判结果:h.264 notebook
来源：百度文库 编辑：中财网 时间：2024/05/03 01:21:52
Q&A:
什么时候用头文件 / include，什么时候用extern？
在.h文件中声明的函数,如果在其对应的.c文件中有定义,那么我们在声明这个函数时,不使用extern修饰符, 如果反之,则必须显示使用extern修饰符.
--http://blog.ednchina.com/tianlebo/23258/message.aspx
当函数提供方单方面修改函数原型时，如果使用方不知情继续沿用原来的extern申明，这样编译时编译器不会报错。但是在运行过程中，因为少了或者多了输入参数，往往会照成系统错误，这种情况应该如何解决？
目前业界针对这种情况的处理没有一个很完美的方案，通常的做法是提供方在自己的xxx_pub.h中提供对外部接口的声明，然后调用方include该头文件，从而省去extern这一步。以避免这种错误。
--http://hi.baidu.com/ice_water/blog/item/2119a06ef9ee27d281cb4a9a.html
总结：尽量用include头文件以避免函数原型在未知情的情况下被修改。头文件中用不用extern因人而异。第一个链接中对应*.c的头文件不要extern（理由是缺省就是extern而且可以知道在那个文件定义），第二个链接中对应*.c的头文件要extern（理由是用不用没有区别而且习惯问题）。
1）所有定义在.c文件中的函数在对应的头文件都不要extern；2）尽量包含头文件而不是用extern以避免函数原型在未知情的情况下被修改；3）实在不想include某个头文件中所有的extern函数（或那个头文件与本文件中的某些定义可能存在冲突？）时用extern
What is the difference between fopen and open in C?
open() is the Unix system call; fopen() is the standard C function
A Unix implementation will use open() in the implementation of fopen(), but the stream returned by fopen() provides buffering and works with functions like printf().
Unless you want to take advantage of system-specific features, stick with fopen().
--http://coding.derkeiler.com/Archive/C_CPP/comp.lang.c/2008-05/msg01629.html
f = fopen (Filename, "wb"); p_in=open(input->infile, OPENFLAGS_READ)
how to use fscanf?
http://topic.csdn.net/u/20071122/22/48282C35-3317-4B3A-B101-77A84C019718.html
My understanding:
我的理解: 因为重载,所以模板
模板屏蔽掉了数据类型
C++核心之一:共享,共享,再共享!!!
Useful Links:
挑战30天 C/C++ 入门极限系列教程：
http://www.pconline.com.cn/pcedu/specialtopic/050514cpp/index.html
back to top
Socket Programming:
|| Reading Notes || Actions || Q&A || My understanding || Useful Links ||
Reading Notes:
for C/C++:
http://www.chinaunix.net/jh/25/48248.html
对于流式套接字你要作的是 send() 发 送数据。对于数据报式套接字，你按照你选择的方式封装数据然后使用 sendto()。
有两种字节排列顺序：重要的字节 (有时叫 "octet"，即八 位位组) 在前面，或者不重要的字节在前面。前一种叫“网络字节顺序 (Network Byte Order)”。有些机器在内部是按照这个顺序储存数据，而另外 一些则不然。当我说某数据必须按照 NBO 顺序，那么你要调用函数(例如 htons() )来将它从本机字节顺序 (Host Byte Order) 转换过来。如果我没有 提到 NBO，那么就让它保持本机字节顺序。
htons()--"Host to Network Short"
htonl()--"Host to Network Long"
ntohs()--"Network to Host Short"
ntohl()--"Network to Host Long"
假设你已经有了一个sockaddr_in结构体ina，你有一个IP地址"132.241.5.10"要储存在其中，你就要用到函数inet_addr(),将IP地址从点数格式转换成无符号长整型。注意，inet_addr()返回的地址已经是网络字节格式，所以你无需再调用 函数htonl()
有没有其相反的方法呢？ 它可以将一个in_addr结构体输出成点数格式？这样的话，你就要用到函数 inet_ntoa().需要注意的是inet_ntoa()将结构体in-addr作为一 个参数
如果你想侦听进入的连接，那么系统调用的顺序可 能是这样
socket();
bind();
listen();
/* accept() 应该在这 */
for python:
http://www.python.org/doc/current/lib/module-socket.html
That raises the question of how the client will know that it has received the entire message sent by the server.
The answer is that recv() will return an empty string when that occurs. And in turn, that will occur when
the server executes close()
recv() will block if no data has been received 9 but the connection has not been closed
recv() will return an empty string when the connection is closed
how to setup server to support multi connection?
by select() (adapted by Windows) and poll() (not yet available in Windows)
Actions:
Q&A:
My understanding:
Useful Links:
My demo: TCP_server TCP_client UDP_server UDP_client
(since the demo is writen in Python, you may download the Python IDE here)
Tutorial on Network Programming with Python.pdf
back to top
H.264:
|| Reading Notes || Actions || Q&A || My understanding || Useful Links ||
Reading Notes:
coded video sequence --> Access units --> NAL unit
auxiliary coded picture: must contain the same number of macroblocks as the primary coded picture.
redundant coded picture: is not required to contain all macroblocks in the primary coded picture.
Flexible macroblock ordering (FMO), also known as slice groups, and arbitrary slice ordering (ASO), which are techniques for restructuring the ordering of the representation of the fundamental regions (macroblocks) in pictures.
--WiKi
Intra mode: 9 for 4*4, 4 for 16*16
For 4x4: In addition to “DC” prediction (where one value is used to predict the entire 4 4 block), eight directional prediction modes are specified as illustrated on the right-hand side of Fig. 10. [Overview of the H.264...]
For 16x16: Predictionmode 0 (vertical prediction),mode 1 (hor-izontal prediction), and mode 2 (DC prediction) are specified similar to the modes in Intra_4 4 prediction except that instead of 4 neighbors on each side to predict a 4 4 block, 16 neighbors on each side to predict a 16 16 block are used. For the specification of prediction mode 4 (plane prediction), please refer to [1]. [Overview of the H.264...]
For a 16×16 luma block, there are 4 other prediction modes: Mode 0 (vertical mode), Mode 1 (horizontal mode) Mode 2 (DC mode) and Mode 4 (plane mode), which is based on a linear spatial interpolation by using the upper and left-hand predictors of the MB.
Inter mode:
Partitions with luma block sizes of 16 16, 16 8, 8 16, and 8 8 samples are supported by the syntax. In case partitions with 8 8 samples are chosen, one additional syntax element for each 8 8 partition is transmitted. This syntax element specifies whether the corresponding 8 8 partition is further partitioned into partitions of 8 4, 4 8, or 4 4 luma samples and corresponding chroma samples. Fig. 12 illustrates the partitioning. [Overview of the H.264/AVC Video Coding Standard]
Acronym:
Coded_Block_Pattern (CBP): Coded Block Pattern
http://ieeexplore.ieee.org/iel5/10549/33371/01579901.pdf
http://en.wikipedia.org/wiki/Macroblock
slice group:
Each slice group can also be divided in several slices
A slice can be decoded independently
each slice is transmitted independently in separate units called packets
http://en.wikipedia.org/wiki/Flexible_Macroblock_Ordering
EBSP: JVT WD [4], Section 8.1.2, defines the encapsulated byte sequence payload (EBSP). EBSP is basically same as RBSP, however, EBSP contains additional bytes for preventing start code emulation. The EBSP format is necessary if the decoder detects slice or picture boundaries by start codes. However, since NAL packets do not use start codes to specify the slice boundaries, the EBSP format is not necessarily used. Use of RBSP is bit efficient and facilitates protocol/format conversion.
对抗块效应滤波器:
应用抗块效应滤波器的目的是为了减少块失真。抗块效应滤波器是在编码器和解码器的反变换之后应用的。滤波器有两种好处：（a）平滑块边缘,改善解码图像质量（特别是在较高的压缩比时）；（b）为了在编码器中对后面的帧进行运动补偿预测，使用滤波宏块，造成预测后产生一个较小的残差。操作过程是这样的：对帧内编码宏块进行滤波，使用未滤波的重建宏块形成预测帧，进行帧内预测，但整幅图像边缘不被滤波。
Actions:
Q&A:
what is the relationship among slice, RBSP and NAL?
My understanding:
rate_control:
same QP:
/ same quantization distortion
higher motion --> bigger residue data - higher datarate
\ higher channel distortion
adaptive QP(bigger QP for bigger residue data and motion vector):
/ higher quantization distortion
higher motion --> bigger residue data - almost same datarate
\ higher channel distortion
Useful Links:
Rate Control and H.264: http://www.pixeltools.com/rate_control_paper.html
JM online documents:
encoder: http://iphome.hhi.de/suehring/tml/doc/lenc/html/index.html
decoder: http://iphome.hhi.de/suehring/tml/doc/ldec/html/index.html
T264: H.264网络传输中马赛克问题的解决
video sequence: http://yufeng1684.bokee.com/6760108.html or 转载
back to top
JM14.0:
|| Reading Notes || Actions || Q&A || My understanding || Useful Links ||
Reading Notes:
-------------------------encoder-----------------------------------------
encoder architecture
encoding:
main()-->encode_one_frame()-->frame_picture()--code_a_picture(frame)-->encode_one_slice()-->start_macroblock()/encode_one_macroblock()
void (*encode_one_macroblock) (Macroblock *currMB)==>void encode_one_macroblock_high (Macroblock *currMB);
compute_mode_RD_cost(mode, currMB, enc_mb, &min_rdcost, &min_dcost, &min_rate, i16mode, bslice, &inter_skip);
packetization:
main()-->encode_one_frame()-->writeout_picture()-->writeUnit()
save decoded picture buffer (DPB):
main()-->encode_one_frame()-->store_picture_in_dpb(enc_frame_picture[0])-->insert_picture_in_dpb(dpb.fs[dpb.used_size],p)
in dump_dpb(), we may set DUMP_DPB = 1 to get debug information!
write test_rec.yuv:
main()-->flush_dpb()-->output_one_frame_from_dpb()-->write_stored_frame(dpb.fs[pos], p_dec)-->write_picture(fs->frame, p_out, FRAME)-->write_out_picture(p, p_out)
-->img2buf (p->imgY, buf, p->size_x, p->size_y, symbol_size_in_bytes, crop_left, crop_right, crop_top, crop_bottom)
-->write(p_out, buf, (p->size_y-crop_bottom-crop_top)*(p->size_x-crop_right-crop_left)*symbol_size_in_bytes)
在JM14.0中有两个结构体比较重要：ImageParameters和StorablePicture
在global.h中定义： ImageParameters用于保存程序运算过程的图像参数
1）imgpel mpr [MAX_PLANE][16][16]; 用于保存预测的像素值
2）int m7 [MAX_PLANE][16][16]; 用于保存residue data的处理过程临时数据（注释写的有问题“the diff pixel values between the original macroblock/block and its prediction”）
3）int ****cofAC; / int ***cofDC; 用于保存经过transform和quantization以后的宏块系数
4）Slice *currentSlice;
DataPartition *partArr;
Bitstream *bitstream;
byte *streamBuffer; 用于保存最终的编码结果，输出到test.264
在mbuffer.h中定义：StorablePicture用于保存图像处理的结果
imgpel ** imgY; / imgpel *** imgUV; 用于保存重构图像的像素值，输出到test_rec.yuv
imgpel ** p_curr_img; ??
short **** mv; 用于保存motion vector的值
这两个结构体各定义了一个全局变量用来保存图像的处理结果：
在lencod.c中定义：ImageParameters images, *img = &images;
在image.c中定义：StorablePicture *enc_picture;（其实还定义了StorablePicture **enc_frame_picture;但是enc_frame_picture[i]取决于rd_pass变量对i有不同的取值，在实验中只用到enc_frame_picture[0]）
在zhifeng.c和zhifeng.h两个文件中用到img->mpr，enc_picture->imgY/enc_picture->imgUV和enc_picture->mv来得到所有像素的motion vector和residue data。
---------------------------------------------------
me_fullfast.c中的FastFullPelBlockMotionSearch()函数中：
mcost = block_sad[pos_00[list][ref]] + MV_COST_SMP (lambda_factor, 0, 0, pred_mv[0], pred_mv[1]);
是在根据RDOptimization做rate distortion optimazatin。虽然这里RateControlEnable=0，但是对于每p帧中任一宏块都有7中预测模式，理论上用4×4的block得到的distortion最小，但是rate最大，这条语句就是做两者的balance。
RateControlEnable的设置是对整个video sequence做的。
用-d "*.cfg"不用-f "*.cfg"!!!(-f好像是组合参数，lencod -f "encoder_baseline.cfg"对于没有修改的JM中的lencod运行出错)
可以用Bit_Buffer来计算每一帧的大小，但是好像不能计算slice的大小。
image.c：Bit_Buffer[total_frame_buffer] = (int) (stats->bit_ctr - stats->bit_ctr_n);
original图像：
/ buf = malloc (xs*ys * symbol_size_in_bytes)
image.c-->ReadOneFrame()-->
\ read(p_in, buf, bytes_y)
但是buf的值在ReadOneFrame()最后被释放：free (buf);
通过imgY_org_frm保存的是16位，高8位为0x00：
buf2img(imgY_org_frm_JV[0], buf, xs, ys, symbol_size_in_bytes);
buf2img(imgUV_org_frm[0], buf, xs_cr, ys_cr, symbol_size_in_bytes);
buf2img(imgUV_org_frm[1], buf, xs_cr, ys_cr, symbol_size_in_bytes);
重要参数：
stats->bit_ctr_parametersets_n：SPS和PPS的位数
start_sequence()-->stats->bit_ctr_parametersets_n = len;
encode_one_frame（）-->stats->bit_ctr_parametersets_n=0;
stats->bit_ctr_n: 当前处理的帧之前的所有位数
encode_one_frame（）-->stats->bit_ctr_n = stats->bit_ctr;
stats->bit_ctr: 在ReportFirstframe(）中清零！
ReportFirstframe(）-->stats->bit_ctr = 0;
ok:
output:
.cfg
ReportFrameStats = 0 # (0:Disable Frame Statistics 1: Enable)
DisplayEncParams = 0 # (0:Disable Display of Encoder Params 1: Enable)
Verbose = 1 # level of display verboseness (0:short, 1:normal, 2:detailed)
in dump_dpb(), we may set DUMP_DPB = 1 to get debug information!
全局变量：
frame_pic
defined in global.h: Picture **frame_pic;
*img:
defined in lencod.c: ImageParameters images, *img = &images;
referenced in global.h: extern ImageParameters *img;
difference between JM13 and JM14:
there is no LoopFilterDisable in JM14, but deblocking filter!
-------------------------decoder-----------------------------------------
decoder architecture
运动补偿过程：
预测mv：在预测一个宏块的mv时，相邻的a，b，c若不可用，则置零。
mv_a = block_a.available ? tmp_mv[list][block_a.pos_y][block_a.pos_x][hv] : 0;
mv_b = block_b.available ? tmp_mv[list][block_b.pos_y][block_b.pos_x][hv] : 0;
mv_c = block_c.available ? tmp_mv[list][block_c.pos_y][block_c.pos_x][hv] : 0;
decode_one_slice()-->read_one_macroblock()-->SetMotionVectorPredictor()
读入运动矢量:
curr_mv [k] = (short)(curr_mvd[k] + pred_mv[k]);
decode_one_slice()-->read_one_macroblock()-->readMotionInfoFromNAL()-->readMBMotionVectors()
读入coefficients：
read_one_macroblock()-->readCBPandCoeffsFromNAL()
执行mpr：（并没有加上residue）
memcpy(&(curr_mpr[0][ioff]), &(block[0][0]), hor_block_size * ver_block_size * sizeof(imgpel));
decode_one_slice()-->decode_one_macroblock()-->perform_mc()-->mc_prediction()
执行transform:
inverse4x4()
decode_one_slice()-->decode_one_macroblock()-->iTransform()-->iMBtrans4x4()-->itrans_4x4()
mpr加上residue：
m7[j][i] = iClip1(max_imgpel_value, rshift_rnd_sf((m7[j][i] + ((long)mpr[j][i] << DQ_BITS)), DQ_BITS));
decode_one_slice()-->decode_one_macroblock()-->iTransform()-->iMBtrans4x4()-->itrans_4x4()
mb_type defined in defines.h
enum {
PSKIP = 0,
BSKIP_DIRECT = 0,
P16x16 = 1,
P16x8 = 2,
P8x16 = 3,
SMB8x8 = 4,
SMB8x4 = 5,
SMB4x8 = 6,
SMB4x4 = 7,
P8x8 = 8,
I4MB = 9,
I16MB = 10,
IBLOCK = 11,
SI4MB = 12,
I8MB = 13,
IPCM = 14,
MAXMODE = 15
} MBModeTypes;
计算SNR：
snr->snr[k]=(float)(10*log10(max_pix_value_sqd[k]*(double)((double)(comp_size_x[k])*(comp_size_y[k]) / diff_comp[k])));
计算平均SNR：
snr->snra[k]=(float)(snr->snra[k]*(snr->frame_ctr)+snr->snr[k])/(snr->frame_ctr + 1); // average snr chroma for all frames
read_new_slice()-->init_picture()-->exit_picture()-->store_picture_in_dpb()-->direct_output()-->find_snr()
丢包时的执行过程：
设置标志位：
1）currSlice->dpC_NotPresent =1; @ if ((slice_id_c != slice_id_a)|| (nalu->lost_packets))
read_new_slice()
2) currMB->dpl_flag = 1; @ if (IS_INTER (currMB) && currSlice->dpC_NotPresent )
decode_one_slice()-->read_one_macroblock()-->readCBPandCoeffsFromNAL()
3) cbp = 0; / currMB->cbp = cbp; @ if (currMB->dpl_flag)
decode_one_slice()-->read_one_macroblock()-->readCBPandCoeffsFromNAL()
修复：
read_new_slice()-->init_picture()-->exit_picture()-->ercConcealIntraFrame()-->concealBlocks()-->ercPixConcealIMB()-->pixMeanInterpolateBlock()
error concealment有两种：
intra:
read_new_slice()-->init_picture()-->exit_picture()-->ercConcealIntraFrame()
inter:
read_new_slice()-->init_picture()-->exit_picture()-->ercConcealInterFrame()
根据erc_mvperMB和MVPERMB_THR的比较来选用怎么inter conceal：
if(erc_mvperMB >= MVPERMB_THR)
concealByTrial();
else
concealByCopy();
erc_mvperMB的计算方法：
8x4, 4x8, 4x4用平均值得到8x8的erc_mvperMB
16x16, 16x8, 8x16, 8x8直接赋值
然后累加成一个MB的erc_mvperMB。（若MVPERMB_THR设成8，则MB的平均MVx和MVy的和不能超过2，而mv的一个pixel值为4，所以JM14.0中把MVPERMB_THR设成8，MVx和MVy和不能超过0.5个pixel，基本上不执行concealByCopy（））
decode_one_slice()-->ercWriteMBMODEandMV()
如何根据设置来选择concealment：
if (img->conceal_mode==1)
exit_picture()-->store_picture_in_dpb()-->output_one_frame_from_dpb()-->write_lost_ref_after_idr()-->copy_to_conceal()
但是在write_lost_ref_after_idr()之前poc的获取get_smallest_poc()@（while (dpb.used_size==dpb.size)）好像有问题，在16帧以后才执行。然后dpb.last_output_poc = poc
另外：
dpb.size = getDpbSize()
init_dpb()
size = imin( size, 16);
init_dpb()-->getDpbSize()
Actions:
Q&A:
Why put WriteOneFrameMV() in main() in lencod.c file， while put WriteOneFrameResidue() right after encode_one_macroblock() in encode_one_slice() in the slice.c file?
1) Since img->mpr only contains one Macroblock data, so it can not be put in lencod.c like WriteOneFrameMV() where enc_picture->mv contains one frame data.
img->mpr is changed in encode_one_macroblock_low()-->LumaResidualCoding()/(Line617)-->LumaResidualCoding8x8()-->LumaPrediction(): memcpy(&(curr_mpr[j][block_x]), l0pred, block_size_x * sizeof(imgpel));
2) The best place to put WriteOneFrameResidue() is right after encode_one_macroblock() because enc_picture->imgY/enc_picture->imgUV is changed in three locations:
a) encode_one_macroblock_low()-->LumaResidualCoding()/(Line617)-->LumaResidualCoding8x8()-->pDCT_4x4()/(Line901)<-->dct_4x4()-->SampleReconstruct(): *imgOrg++ = iClip1( max_imgpel_value, rshift_rnd_sf(*m7++, dq_bits) + *imgPred++);
b) encode_one_macroblock_low()-->LumaResidualCoding()/(Line617)-->LumaResidualCoding8x8(): memcpy(&enc_picture->imgY[img->pix_y + j][img->pix_x + mb_x], &img->mpr[0][j][mb_x], 2 * BLOCK_SIZE * sizeof(imgpel));
c) encode_one_macroblock_low()-->LumaResidualCoding()/(Line617): memcpy(&enc_picture->imgY[img->pix_y+j][img->pix_x], img->mpr[0][j], MB_BLOCK_SIZE * sizeof (imgpel));
So, we may put WriteOneFrameResidue() right after LumaResidualCoding(), however to generally support other encode_one_macroblock mrthods, we put WriteOneFrameResidue() right after encode_one_macroblock().
Why not use img->m7 as residue data ouput?
Although img->m7 is used as a temporary variable for processing residue data, but it is the never accurate residue data value. img->m7 is changed in two locations:
a) encode_one_macroblock_low()-->LumaResidualCoding()/(Line617)-->LumaResidualCoding8x8()-->ComputeResidue(): *m7++ = *imgOrg++ - *imgPred++; (where m7 has not been quantized)
b) after forward4x4() and quant_4x4(),
If there is nonzero coefficient, img->m7 is changed in inverse4x4() and the value should be rshift_rnd_sf() to get residue data as in SampleReconstruct().
If there is no nonzero coefficient, img->m7 is invalid because only predicted value is used for enc_picture->p_curr_img.
If no residue data output in the global structure, how does JM encode residue data?
The residue data is needed only after transform and quantization in H.264 decoder, so JM14.0 does not keep original residue data in global variable.
The one used for H.264 decoder is defined in img->cofAC/img->cofDC:
which are changed in quant_4x4(),<-->quant_4x4_around() (int* ACLevel = img->cofAC[b8][b4][0]; defined in dct_4x4())
which are used to be VLC coded in main()-->encode_one_frame()-->frame_picture()-->code_a_picture(frame)-->encode_one_slice()-->write_one_macroblock()-->writeMBLayer()-->writeCoeff16x16()-->writeCoeff8x8()-->writeCoeff4x4_CAVLC()
test_rec.yuv通过什么变量写入？
dpb.fs[pos]: write_stored_frame(dpb.fs[pos], p_dec)
enc_frame_picture[0]：
初始化地址：prepare_enc_frame_picture( &enc_frame_picture[0] )<--frame_picture()
1）通过get_mem2Dpel (&(s->imgY), size_y, size_x)，给s->imgY赋地址，s是alloc_storable_picture的返回值。
2）通过(*stored_pic) = alloc_storable_picture ((PictureStructure) img->structure, img->width, img->height, img->width_cr, img->height_cr)给函数prepare_enc_frame_picture的参数StorablePicture **stored_pic赋地址
注意：不是分配内存，所以不用对enc_frame_picture[0]赋值，而是用enc_frame_picture[0]作为函数prepare_enc_frame_picture的参数来赋地址！
写地址内容：encode_one_macroblock_high()-->compute_mode_RD_cost()-->RDCost_for_macroblocks()
-->Intra16x16_Mode_Decision()-->currMB->cbp = dct_16x16 (currMB, PLANE_Y, *i16mode)-->img_Y[i] = iClip1( max_imgpel_value, rshift_rnd_sf(M1[j][i], DQ_BITS) + predY[i])这里一次向&enc_frame_picture[0]写两个字节的内容
compute_mode_RD_cost（）一次写16次（的两个字节）
-->currMB->cbp = Mode_Decision_for_Intra4x4Macroblock (currMB, lambda, &dummy_d)-->Mode_Decision_for_8x8IntraBlocks()-->Mode_Decision_for_4x4IntraBlocks()-->RDCost_for_4x4IntraBlocks()
-->*nonzero = pDCT_4x4 (currMB, PLANE_Y, block_x, block_y, &dummy, 1)-->orig_img[i] = iClip1( max_imgpel_value, rshift_rnd_sf(m7[i], DQ_BITS) + pred_img[i])
在insert_picture_in_dpb(dpb.fs[dpb.used_size],p)中，将dpb.fs指向enc_frame_picture[0]
main()-->encode_one_frame()-->store_picture_in_dpb(enc_frame_picture[0])-->insert_picture_in_dpb(dpb.fs[dpb.used_size],p)
打开文件：p_dec=open(params->ReconFile, OPENFLAGS_WRITE, OPEN_PERMISSIONS)
main()-->Configure()-->PatchInp()
关闭文件：close(p_dec)
main()
test.264通过什么变量写入？
frame_pic[img->rd_pass]:
初始化：
定义在global.h的全局变量 Picture **frame_pic;
将frame_pic[0]赋给frame：frame_picture (frame_pic[0], 0)
main()-->encode_one_frame()-->frame_picture()
将frame/frame_pic[0]赋给pic：code_a_picture(frame)；
main()-->encode_one_frame()-->frame_picture()--code_a_picture(frame)
在函数encode_one_slice()中pic参数只被赋给img->currentPicture: img->currentPicture = pic
将pic/frame/frame_pic[0]赋给img->currentPicture：img->currentPicture = pic
img是全局变量：ImageParameters images, *img (lencod.c) = &images; / extern ImageParameters *img; (global.h)
将img->currentPicture赋给currPic: Picture *currPic = img->currentPicture;
在init_slice()中分配内存：currPic->slices[currPic->no_slices-1] = malloc_slice();
即: frame_pic[0]->slices[currPic->no_slices-1] = malloc_slice() / img->currentPicture->slices[currPic->no_slices-1] = malloc_slice()
main()-->encode_one_frame()-->frame_picture()-->code_a_picture(frame)-->encode_one_slice()-->init_slice()
写内容：
currStream->streamBuffer[currStream->byte_pos++] = currStream->byte_buf
encode_one_macroblock_high()-->submacroblock_mode_decision()-->RDCost_for_8x8blocks()-->writeCoeff8x8()-->writeCoeff4x4_CAVLC()
-->writeSyntaxElement_NumCoeffTrailingOnes()-->writeUVLC2buffer()
-->writeSyntaxElement_Level_VLC1()/writeSyntaxElement_Level_VLCN()-->writeUVLC2buffer()
-->writeSyntaxElement_TotalZeros()-->writeUVLC2buffer()
encode_one_macroblock_high()-->compute_mode_RD_cost()/共(max_index = 9)次-->RDCost_for_macroblocks()-->writeMBLayer()-->writeCoeff16x16()-->writeCoeff8x8()-->writeCoeff4x4_CAVLC()
写到test.264：
currSlice = pic->slices[slice] 这里currSlice通过行参pic指向writeout_picture (frame_pic[img->rd_pass])实参frame_pic[img->rd_pass]: currSlice = pic->slices[slice] (=frame_pic[img->rd_pass]->slices[slice])
currStream = (currSlice->partArr[partition]).bitstream （=(frame_pic[img->rd_pass]->slices[slice]->partArr[partition]).bitstream）
main()-->encode_one_frame()-->writeout_picture()
memcpy (&nalu->buf[1], currStream->streamBuffer, nalu->len-1); 然后：WriteNALU (nalu)
main()-->encode_one_frame()-->writeout_picture()-->writeUnit()
打开文件：f = fopen (Filename, "wb")
main()-->start_sequence()-->OpenAnnexbFile()
关闭文件：fclose (f)
main()-->terminate_sequence()-->CloseAnnexbFile()
在哪里估计mv?
1)
mv[0] = offset_x + spiral_search_x[best_pos];
mv[1] = offset_y + spiral_search_y[best_pos];
encode_one_macroblock_low()-->PartitionMotionSearch()
encode_one_macroblock_low()-->submacroblock_mode_decision()-->PartitionMotionSearch()
-->BlockMotionSearch()-->IntPelME()<-->FastFullPelBlockMotionSearch()
2)
然后，skip模式可能重写mv：
mv[0] = img->all_mv [0][0][0][0][0][0];
mv[1] = img->all_mv [0][0][0][0][0][1];
encode_one_macroblock_low()-->PartitionMotionSearch()-->BlockMotionSearch()
encode_one_macroblock_low()-->submacroblock_mode_decision()-->PartitionMotionSearch()-->BlockMotionSearch()
3)
关于mv的内容： short*** all_mv = &img->all_mv[list][ref][blocktype][block_y]; //!< block type (1-16x16 ... 7-4x4)
all_mv[0][i][0] = mv[0];
all_mv[0][i][1] = mv[1];
encode_one_macroblock_low()-->PartitionMotionSearch()-->BlockMotionSearch()
encode_one_macroblock_low()-->submacroblock_mode_decision()-->PartitionMotionSearch()-->BlockMotionSearch()
PSliceSkip设置：enc_mb->valid[0] = (!intra && params->InterSearch[bslice][0])
encode_one_macroblock_low()-->init_enc_mb_params()
img->all_mv
分配内存：
int get_mem_mv (short ******* mv)
{
// LIST, reference, block_type, block_y, block_x, component
get_mem6Dshort(mv, 2, img->max_num_references, 9, 4, 4, 2);
return 2 * img->max_num_references * 9 * 4 * 4 * 2 * sizeof(short);
}
get_mem_mv (&(img->pred_mv));
get_mem_mv (&(img->all_mv));
main()->init_img()
enc_picture->mv
初始化
写内容：通过img->all_mv写入enc_picture->mv
memcpy(enc_picture->mv [LIST_0][block_y][block_x + i], img->all_mv[LIST_0][best_l0_ref][mode][j][i], 2 * sizeof(short))
encode_one_macroblock_low()-->assign_enc_picture_params（）
enc_picture->mv[LIST_0][by][bx][0] = all_mv [LIST_0][ ref][mode8][j][i][0];
enc_picture->mv[LIST_0][by][bx][1] = all_mv [LIST_0][ ref][mode8][j][i][1];
encode_one_macroblock_low()-->SetMotionVectorsMB (currMB, bslice)
enc_frame_picture[0]
什么时候写出mv？
在encode_one_frame()之后
mv在什么时候变化?
init_frame ()
main()-->encode_one_frame()
什么时候写出residue？
//===== S E T F I N A L M A C R O B L O C K P A R A M E T E R S ======
通过(*curr_mpr)[16] = img->mpr[0]
memcpy(&(curr_mpr[j][block_x]), l0pred, block_size_x * sizeof(imgpel));
通过m7 = &img_m7[j][mb_x]：*m7++ = *imgOrg++ - *imgPred++;
encode_one_macroblock_low()-->LumaResidualCoding()-->LumaResidualCoding8x8()-->ComputeResidue()/Line863
//===== DCT, Quantization, inverse Quantization, IDCT, Reconstruction =====
通过img->m7：
(*curr_res)[MB_BLOCK_SIZE] = img->m7[pl];
m7 = &img_m7[j][mb_x];
SampleReconstruct (imgpel **curImg, imgpel mpr[16][16], int img_m7[16][16], int mb_y, int mb_x, int opix_y, int opix_x, int width, int height, int max_imgpel_value, int dq_bits)
*imgOrg++ = iClip1( max_imgpel_value, rshift_rnd_sf(*m7++, dq_bits) + *imgPred++);
encode_one_macroblock_low()-->LumaResidualCoding()/(Line617)-->LumaResidualCoding8x8()-->pDCT_4x4()/(Line901)<-->dct_4x4()-->SampleReconstruct()
encode_one_macroblock_low()-->ChromaResidualCoding()/(Line645)
/*!
计算宏块的残差m7：
*m7++ = *imgOrg++ - *imgPred++;
encode_one_macroblock_low()-->submacroblock_mode_decision()-->LumaResidualCoding8x8()-->ComputeResidue()
m7_line = &m7[j][block_x];
*m7_line++ = (int) (*cur_line++ - *prd_line++)
encode_one_macroblock_low()-->Mode_Decision_for_Intra4x4Macroblock()-->Mode_Decision_for_8x8IntraBlocks()-->Mode_Decision_for_4x4IntraBlocks_JM_Low()-->generate_pred_error()
img->m7[uv + 1][j][i] = imgUV_org[uv][img->opix_c_y+j][img->opix_c_x+i] - curr_mpr[j][i]; （在global.h中定义imgpel ***imgUV_org;）
encode_one_macroblock_low()-->ChromaResidualCoding()/line645
*/
几种intrapred：
encode_one_macroblock_low()-->Mode_Decision_for_Intra4x4Macroblock()-->Mode_Decision_for_8x8IntraBlocks()-->Mode_Decision_for_4x4IntraBlocks_JM_Low()-->intrapred_4x4()
encode_one_macroblock_low()-->Mode_Decision_for_new_Intra8x8Macroblock()-->(point functions below)-->Mode_Decision_for_new_8x8IntraBlocks_JM_High()/Mode_Decision_for_new_8x8IntraBlocks_JM_Low()-->intrapred_8x8()
encode_one_macroblock_high()/encode_one_macroblock_highfast()/encode_one_macroblock_highloss()-->compute_mode_RD_cost()-->RDCost_for_macroblocks()-->Mode_Decision_for_new_Intra8x8Macroblock()-->(above)-->Mode_Decision_for_new_8x8IntraBlocks_JM_High()/Mode_Decision_for_new_8x8IntraBlocks_JM_Low()-->intrapred_8x8()
Mode_Decision_for_new_8x8IntraBlocks = Mode_Decision_for_new_8x8IntraBlocks_JM_Low;
Mode_Decision_for_new_8x8IntraBlocks = Mode_Decision_for_new_8x8IntraBlocks_JM_High;
encode_one_macroblock_low()-->intrapred_16x16()
encode_one_macroblock_high()/encode_one_macroblock_highfast()/encode_one_macroblock_highloss()-->compute_mode_RD_cost()-->RDCost_for_macroblocks()-->Intra16x16_Mode_Decision()-->intrapred_16x16()
？？为什么不能用img->m7来保存residue（就算没有post-scaling, the last step(8-356) of 8.5.10）？
因为在LumaResidualCoding8x8()中pDCT_4x4执行后，还有一行修改了enc_picture->imgY值，所以m7并不是residue：
memcpy(&enc_picture->imgY[img->pix_y + j][img->pix_x + mb_x], &img->mpr[0][j][mb_x], 2 * BLOCK_SIZE * sizeof(imgpel));
encode_one_macroblock_low()-->submacroblock_mode_decision()-->LumaResidualCoding8x8()
-------------------------decoder-----------------------------------------
？总是会执行exit_picture()中的DeblockPicture( img, dec_picture );
不是，在函数内部判断
如何在处理完一帧后，进行error concealment？
在init_picture()中：
if (dec_picture)
{
// this may only happen on slice loss
exit_picture();
}
read_new_slice()-->init_picture()
应该放到decode_one_frame()中的最后！！！但是因为while ((currSlice->next_header != EOS && currSlice->next_header != SOP))一直为真，所以并没有被执行到。而是从if (current_header == EOS)内退出了
My understanding:
对JM的建议：
1）应该增加一个标志位在MB中标明哪几个8x8的subMB的coeff被置成零！因为几个地方置coeff，而img->m7没有相应被修改，导致很难得到residue data。
有两个地方用到LumaResidualCoding8x8()，这里第二次需要重新reset_residuex_zero()
encode_one_macroblock_low()-->submacroblock_mode_decision()-->LumaResidualCoding8x8()
encode_one_macroblock_low()-->LumaResidualCoding()-->LumaResidualCoding8x8()
有两个地方用mpr的值重设imgY：
memcpy(&enc_picture->imgY[img->pix_y+j][img->pix_x], tr8x8.mpr8x8[j], MB_BLOCK_SIZE * sizeof(imgpel));
或者：memcpy (&enc_picture->imgY[img->pix_y+j][img->pix_x],tr4x4.mpr8x8[j], MB_BLOCK_SIZE * sizeof(imgpel));
encode_one_macroblock_low()-->SetCoeffAndReconstruction8x8()
memcpy(&enc_picture->imgY[img->pix_y+j][img->pix_x], img->mpr[0][j], MB_BLOCK_SIZE * sizeof (imgpel));
encode_one_macroblock_low()-->LumaResidualCoding()
-------------------------decoder-----------------------------------------
？当partitioning C的RTP包不是在partitioning A的RTP包之后到达时，解码器会有问题？（在NALU_TYPE_DPA内部执行read_next_nalu(nalu)）
对JM的建议：
1）MVPERMB_THR的取值过小（JM14.0中用8，在erc_api.h中）
2）erc_mvperMB的计算有问题：
不能用erc_mvperMB /= dec_picture->PicSizeInMbs;
因为在decode_one_slice()-->ercWriteMBMODEandMV()中累加erc_mvperMB，而丢失partitionA包时不执行decode_one_slice()
erc_mvperMB += iabs(pRegion->mv[0]) + iabs(pRegion->mv[1]);
read_new_slice()-->init_picture()-->exit_picture()
Useful Links:
0) for beginner:
http://www.h263l.com/h264/h264_overview.pdf
1) JM14.0 source code:
http://iphome.hhi.de/suehring/tml/download/
2) H.264 standard:
http://www.itu.int/rec/T-REC-H.264
3) H.264 Wiki:
http://en.wikipedia.org/wiki/H.264
4) JM online document:
http://iphome.hhi.de/suehring/tml/doc/lenc/html/index.html
http://iphome.hhi.de/suehring/tml/doc/ldec/html/index.html
5) JM manual:
http://iphome.hhi.de/suehring/tml/JM%20Reference%20Software%20Manual%20(JVT-X072).pdf
6) Overview of the H.264/AVC Video Coding Standard
http://ieeexplore.ieee.org/iel5/76/27384/01218189.pdf
7) H264Visa.exe
8) YUVviewer.exe
9) RTP Payload Format for H.264 Video
http://tools.ietf.org/html/rfc3984
10) x264
http://blog.csdn.net/sunshine1314/archive/2005/05/20/377158.aspx
http://lspbeyond.go1.icpcn.com/x264/index.htm
11) white paper
http://www.vcodex.com/h264.html(in sections)
http://bbs.chinavideo.org/viewthread.php?tid=3336&extra=page%3D1(whole)
back to top
FFMPEG:
|| Reading Notes || Actions || Q&A || My understanding || Useful Links ||
Reading Notes:
How to support camera in windows for FFMPEG？
ffmpeg -r 15 -f vfwcap -i 0 output.mpg
http://ffmpeg.arrozcru.org/forum/viewtopic.php?f=8&t=763